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FOREWORD 


This  technical  report  describes  work  conducted  as  part  of  the  Navy  Personnel  Research  and 
Development  Center’s  Communication  Networks  in  Training  (CNIT)  project  in  the  general  area  of 
remote-site  training.  The  CNIT  project  is  one  part  of  the  Schoolhouse  Training  product  line  and 
falls  under  the  Personnel  and  Training  Technology  (NP2A)  Block  of  the  6.2  Mission  Support 
Technology  Program  Element  0602233,  Woric  Unit  RM33T23.02.  The  work  was  performed  under 
the  sponsorship  of  the  Office  of  Naval  Technology.  The  objective  of  the  project  is  to  find  more  cost- 
effective  ways  to  train  personnel  who  are  geographically  remote  from  training  resources.  The 
project  has  been  exploring  the  use  of  new  communication  technologies  to  export  training  to 
geographically-remote  students.  Among  these  technologies  pj  i  computer  networking,  instructional 
TV,  videotape,  audiographics,  videographics,  and  other  media.  This  technical  report  describes  the 
results  of  a  6-month  laboratory  study  involving  743  Navy  students,  which  investigated  the  relative 
training  effectiveness  and  user  acceptance  of  live  instruction,  2-way  video,  1-way  video,  and 
audiographics.  The  findings  have  direct  implications  for  the  design  of  future  distance  education 
systems  in  the  Navy  and  elsewhere. 

The  recommendations  in  this  technical  report  are  intended  for  use  by  the  Chief  of  Naval 
Education  and  Training  and  Chief  of  Naval  Operations  (OP- 11)  in  developing  policy  for  the 
application  of  advanced  communication  technology  in  the  Navy. 


THOMAS  F.  FINLEY 
Captain,  U.S.  Navy 
Commanding  Officer 


RICHARD  C.  SORENSON 
Technical  Director  (Acting) 
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SUMMARY 


Problem  and  Background 

Because  of  the  wide  geopaphic  dispersal  of  home  ports,  fleet  units,  and  Navy  Reserve 
detachments,  it  is  expensive  to  transport  Navy  personnel  to  a  few  facilities  for  classroom  training. 
The  costs  involve  transportation,  travel  expenses,  and  the  travel  time  lost  from  duty.  New 
communication  technologies  have  the  potential  to  reduce  the  cost  of  training.  For  example. 
Videoteletraining  (VTT)  can  be  used  to  deliver  instruction  with  instructional  TV  and  may  link 
students  and  instructor  across  great  distances.  There  are  alternative  VTT  technologies  that  vary 
greatly  in  cost.  Ihe  solution  to  the  Navy’s  remote-site  training  problem  lies  in  the  prc^r  selection 
and  use  of  the  most  cost-effective  communication  technologies  such  as  VTT. 

Objectives 

The  overall  objective  of  the  project  is  to  explore  technologically  cost-effective  ways  to  train 
personnel  who  are  geographically  remote  ftom  training  resources.  The  objective  of  the  work 
described  in  this  report  was  to  conduct  an  empirical  study  comparing  (1)  training  effectiveness  and 
(2)  user  acceptance  of  live  instruction  and  six  different  alternative  VTT  technologies:  multi¬ 
channel  2-way  vide^  with  2-way  audio,  single  channel  2- way  video  with  2- way  audio,  1-way  video 
with  2-way  audio,  1-way  video  with  1-way  audio,  i-way  video  with  intermittent  2- way  audio, 
and  audiographiCb. 

Method 

The  method  included  steps  to  (1)  define  assumptions  and  simulation  requirements,  (2)  develop 
a  research  plan,  (3)  prepare  for  VTT  training,  (4)  design  and  install  a  VTT  laboratory,  and  (5) 
collect  and  analyze  data. 

Results 

VTT,  in  several  different  forms,  was  efFectivc  both  in  terms  of  student  perfomtance  and  student 
aiid  instructor  acceptance.  The  type  of  VTT  technology  did  influence  student  peifomiancc  and 
attitudes,  but  had  a  far  smaller  effect  than  student  experience. 

The  most  successful  VIT  technologies  were  those  allowing  continuous  2-way  audio 
communication  between  classrooms  with  either  2-way  or  1-way  v^dco.  Using  2- way  video  does 
not  appear  to  improve  student  pctfomiancc  as  con^syared  to  1-w.  v  i-teo  but  instruciOTS  prefer  2- 
way  video  and  students  expressed  the  desire  to  sec  tlu^ir  cohorts  in  oiner  classes,  which  requires  2- 
way  video. 

Student  test  performance  w'as  poorer  with  VTT  systems  that  restricted  remote  students’  ability 
to  converse  with  or  see  the  instructor  and  the  performance  decrement  was  evident  in  both  local  and 
remote  classrooms.  Evidence  suggests  that  student  acceptance  of  such  VIT  technologies  was 
lower  than  w'ith  fully-iiucraciivc  VTT.  Similar  rcsulu  would  be  expected  with  videotaped 
instruction.  Students  adapted  to  compensate  for  the  video  and  audio  shortcomings  of  VTT 
technologies. 
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The  most  serious  shortcoming  of  the  simulated  VTT  technologies  was  audio.  Additional  work 
needs  to  be  done  to  refine  the  audio  systems  and  procedures  used  in  VTT. 

Recommendations 

1.  The  Chief  of  Naval  Education  and  Training  (CNET)  and  the  Naval  Education  and  IVaining 
Program  Management  Support  Activity  should  continue  efforts  to  refine  the  CNET  VTT  netwmk. 

2.  The  Chief  of  Naval  Education  and  Training  and  the  Naval  Education  and  Training  Program 
Management  Support  Activity  should  analyze  the  feasibility  and  cost-effectiveness  of  extending 
the  architecture  of  the  CNET  VTT  network  using  VTT  technologies  such  as  1-way  video  with  2- 
way  audio  and  1-way  video  with  1-way  audio. 

3.  The  Chief  of  Naval  Education  and  Training  and  the  Naval  Education  and  Training  Program 
Management  and  Support  Activity  should  originate  Problem  Description  and  Need  Justifications 
to  continue  the  investigation  of  the  applicability  of  VTT  beyond  lecture-based  courses  (e.g.,  in 
courses  using  “hands-on”  laboratories,  small-group,  and  other  training  processes). 
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INTRODUCTION 


Problem  and  Background 

A  requirement  exists  to  train  Navy  personnel  who  are  geographically  remote  from  training 
resources.  This  requirement  exists  throughout  the  Navy,  but  is  perhaps  most  obvious  for  personnel 
aboard  ships  at  sea.  Shipboard  training  is  limited  by  available  training  resources  and  the  skills  of 
shipboard  trainers.  By  necessity,  personnel  are  periodically  assigned  to  formal  schools  to  receive 
training  they  cannot  receive  aboard  ship.  The  locations  of  existing  training  facilities  often  require 
fleet  personnel  to  travel  away  from  their  home  duty  station  to  complete  required  training.  The 
remote-site  training  requirement  also  exists  in  the  Navy  reserves.  Reservists  typically  belong  to 
small  detachments,  widely  dispersed  geographically,  with  limited  training  resources,  few  qualified 
trainers,  and  little  time  to  train.  The  requirement  to  overcome  geographic  distance  in  training 
delivery  is  a  generic  problem  that  exists  in  civilian  public  education  and  industry  as  well  as  the 
military. 

Evolving  communication  technologies  have  the  potential  to  reduce  the  impact  of  geography  on 
training.  Compressed-bandwidth  TV,  for  example,  can  link  instructors  and  students  across  distance 
and  permit  travel  to  occur  electronically  rather  than  physically.  Several  other  technologies  can  also 
bridge  distance.  These  include  videotape,  audiographics  systems,  electronic  mail,  audio 
conferencing,  computer-aided  instruction,  and  paper  media.  A  solution  to  the  Navy’s  remote-site 
training  problem  lies  in  the  proper  selection  and  use  of  new  communication  technologies.  In 
general,  these  technologies  are  costly  and  constantly  changing.  Many  forms  of  communication 
technologies  are  being  used  on  a  regular  basis,  and  in  demonstration  projects.  Investigators  are 
exploring  strengths  and  limitations,  cost-effectiveness,  and  other  dimensions  governing  suitability 
for  different  applications.  Unfortunately,  there  is  no  road  map  or  checklist  to  follow  to  detemiine 
which  technology  is  “best”  in  a  particular  application  today  and  which  will  be  “best”  next  week, 
next  year,  or  10  years  from  nov  The  Communication  Networks  in  Training  (CNIT)  project  is 
exploring  different  technologies,  research  and  development  projects,  and  the  Navy’s  training 
problems  to  gain  a  better  understanding  of  which  technologies  hold  the  greatest  potential  for  future 
use  in  the  Navy. 

Objectives 

Ttic  primary  objective  of  the  CNIT  project  is  to  find  more  cost-cffcciivc  ways  to  train  personnel 
who  arc  geographically  remote  from  training  resoumes.  This  objective  is  being  addressed  along 
four  different  tracks: 

1.  Assess  the  applicability  of  new  contniunic'  ion  tcdutologics  to  Uic  solution  of  Navy 
training  problems. 

2.  Design,  develop,  and  evaluate  an  experimental,  computer-based  instructional  support 
network. 

3.  Design,  develop,  and  evaluate  an  cxpcritttcntal,  2-w'ay  vidcotelciraining  (VTT)  system.* 


*Thc  term  ■'vjdcotctcuainir.g*  derives  from  “vtdcotcEcoonfcrcncing.”  a  icchDology  iRitially  used  for  conducting 
TV'  confcr(u«,.cs.  The  ?cdino!egy  u.scs  digital  video  campresiioa  techniquc-s  to  sSSow  communieaiion  via  2-tt’ay  TV' 
widiin  naiTowTcr  banduiddis  titan  arc  ased  in  irsdiiiotal  instructtostal  TV'.  V'TT  Ls  an  c^oluiionary  form  of  msm^tional 
TV. 
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4.  Investigate  the  impact  of  alternative  equipment  configurations  and  training  protocols  on 
training  effectiveness  in  a  VTT  laboratory 

Work  on  tracks  1,  2,  and  3  was  completed  during  FY89  and  is  reponed  in  Simpson  (1990), 
Simpson  &  Pugh  (1990),  and  Simpson,  Pugh,  &  Parchman  (1990).  A  field  survey  of  VTT  systems 
in  public  education,  industry,  and  the  military  was  conducted  as  a  prelude  to  experimental  work  on 
track  4  and  is  reponed  in  Pugh,  Parchman,  &  Simpson  (1991).  The  present  technical  report 
describes  experimental  work  performed  on  track  4  during  FY91. 

Research  Issues 

Several  different  technologies  can  be  used  to  deliver  instruction  to  students  at  a  distance.  The 
technologies  vary  in  capability  and  cost.  They  may  also  vary  in  instructional  effectiveness  and 
acceptance  by  instructors  and  students.  The  impact  of  technology  on  learning  and  attitude  factors 
is  not  adequately  understood. 

Distance  Education  Technologies 

Distance  education  technologies  enable  instruction  to  occur  over  geographic  distance,  with  the 
instructor  at  one  location  and  students  at  anot'  or.  S,  ■cral  such  technologies  are  available,  lliose 
most  suitable  for  Navy  training  enable  groups  o*  soidenis  to  be  trained  simultaneously  in  an 
organized  classroom  situation  much  like  a  iivc  classroom  and,  ideally,  allow  instructor  and 
students  to  participate  synchronously  (i.e.,  instructor  teaches  and  students  learn  at  the  same  lime). 
Based  on  this  dual  standard,  such  technologies  as  electronic  mail,  computer-aided  instniction,  and 
paper  media  may  be  excluded.  The  remaining  candidates  are  various  forms  of  instructional  TV. 
audiographics,  and  videotape.**  These  technologies  vary  in  terms  of  their  surface  features,  potential 
for  instmetor-student  interaction,  complexity,  and  cost.  All  have  been  used  to  deliver  instruction 
effectively  in  public  education,  though  most  have  been  used  only  to  a  limited  degree  in  Navy 
training.  Research  has  shown  that  2-way  TV  is  an  effective  method  to  deliver  Navy  training 
(Rupin.ski  &.  Sioloff.  1990;  Simpson.  Pugh,  &  Parchman.  1990).  IIjc  Chief  of  Naval  Education  and 
Training’s  \nT  Implementation  and  Management  plan  (CNET,  1990.  June)  aivisions  the  eventual 
insiallaiion  of  a  nationwide,  muhitxmu,  satellite-based  network  for  delivering  training  to 
Navy  personnel.  Tltis  system  would  be  modeled  on  CNET’s  prototype  Electronic  Schoolhouse 
Network  (CI-S.N).  which  is  hcadquancrcd  at  Norfolk.  V.A.  at  Meet  Combat  Training  Center. 
Atlantic,  and  with  ncxles  at  Norfolk  and  Dam  Neck,  VA.  Qurlcston.  SC,  and  Mayport  a«id 
Pensacola,  Id*.  1'hc  network  uses  2-way  video;  that  is.  all  sites  on  the  network  can  botii  broadcast 
and  receive  TV.  CNEl  's  future  training  network,  nxsdclcd  on  (he  prototype,  would  be  primarily  a 
2-way  system,  although  it  w^ould  also  include  sotne  ra'ciw-cmiy  sites;  that  is.  sites  that  would  not 
be  able  to  transtnit  TV. 

ln,str\tctional  I'V  has  been  used  in  public  education  for  decades.  Cunemly.  there  aic  hundreds 
of  .such  .systems,  ranging  frotn  the  simplest  single-campus  closed-circuit  T\^  systems  to  nationwide 
networks  linking  dozens  of  campu.ses  (U.S.  Congrcs.s.  1989).  Most  of  these  .systents  use  I -way 
video;  that  is.  studcnt.s  at  remote  sites  can  see  tire  insta*ctor  at  the  originating  or  “local'*  site,  but  the 
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instructor  cannot  see  the  students  (Pugh,  Parchman,  &  Simpson,  1991).  This  is  especially  true 
when  there  are  many  students  at  multiple  remote  sites.  Return  audio  links  from  remote  sites  to  the 
instmctor  may  be  provided  (usually  via  telephone)  to  permit  students  to  ask  questions  and  interact 
with  the  instructor.  The  Army’s  Satellite  Education  Network,  based  at  Ft.  Lee,  VA,  is  built  on  this 
model,  and  has  been  successfully  used  for  several  years  to  deliver  courses  in  contracting,  logistics, 
and  other  topics  (Brockwell,  1989). 

In  some  cases,  instructional  TV  systems  are  entirely  1-way  and  students  can  only  watch  and 
listen,  without  being  seen  or  heard  by  the  instructor.  Videotapes  are  being  used  increasingly  to 
support  live  instruction,  although  there  have  been  few  attempts  to  use  videotape  to  supplant  live 
instmction  altogether.  Audiographics  systems  are  a  relatively  low-cost  alternative  to  instructional 
TV.  These  systems  typically  link  sites  via  telephone  lines  and  permit  each  site  to  transmit  and 
receive  audio  and  still -frame  graphics  that  can  be  annotated  (Pugh,  Parchman,  &  Simpson,  1991). 
Table  1  summarizes  the  attributes  of  several  alternative  distance  education  technologies.  Each  of 
these  technologies  is  discussed  in  greater  detail  below. 

Table  1 

Attributes  of  Alternative  Distance  Education  Technologies 


System 

Type 

Students  Sec  & 
Hear  Instructor 

Students  See  Instructor  Sees 
Graphics  Students 

Instmctor  Hears 
Students 

Synchronous 

Participation 

2V/2A 

+ 

+  + 

+ 

+ 

Multi-channel 

2V/2A+ 

+  (continuous) 

+  (continuous)  +  (conunuous) 

+ 

+ 

1V/2A 

+ 

+ 

+ 

+ 

IV/IA 

+ 

+ 

- 

+ 

Videotape 

-h 

+ 

- 

- 

Audiogiaphics 

- 

(continuous) 

+ 

+ 

Two-Way  Video  with  2-Way  Audio  (2V/2A).  2V/2A  is  primarily  a  conferencing  medium, 
but  is  being  used  increasingly  in  education.  The  architecture  of  2V/2A  systems  is  illustrated  in 
schematic  form  in  Figure  1.  As  shown  in  Table  1, 2V/2A  systems  permit  students  to  see  and  hear 
instructor  and  instructor  to  see  and  hear  students.  Sites  have  multiple  cameras  and  can  switch 
among  them.  Thus,  the  local  site  can  switch  its  output  between  an  instructor  camera  and  a  graphics 
camera,  but  can  only  send  one  channel  at  a  time.  The  remote  (receiving)  sites  can  switch  output 
between  a  student  camera  and  a  graphics  camera,  but  can  only  send  one  channel  at  a  time.  Tliere 
are  generally  one  local  site  and  one  or  more  remote  sites.  Sites  may  have  additional  cameras  and 
be  able  to  switch  to  send  them  out  as  well.  Audio  is  continuous  and  any  person  speaking  at  either 
site  may  be  heard  by  people  at  other  sites,  although  there  are  generally  rules  governing  when  to 
speak  and  student  microphones  may  be  muted  at  f'ertain  times.  Insmictor  and  students  participate 
in  training  synchronously.  2V/2A  has  been  used  successfully  for  Navy  training  in  the  CESN,  die 
west  coast  VTT  demonstration  project,  and  in  public  education. 


Multi-Channel  2-Wr’'  Video  with  2- Way  Audio  (2V/2A+).  2V/2A+  is  a  logical  extension 
of  2V/2A  in  which  each  \  jc  >urce  is  transmitted  on  a  separate  channel  instead  of  switched  on  a 
single  channel.  Thus,  for  example,  a  remote  class  would  observe  the  instructor  on  one  video 
display,  graphics  on  a  second,  and  the  local  class  on  a  third.  Such  an  arrangement  permits  the 
remote  class  environment  to  more  closely  approximate  that  of  the  local  class.  The  architectme  of 
such  a  system  is  illustrated  in  schematic  form  in  Figure  2.  As  shown  in  Table  1,  the  attributes  of  a 
2V/2A  system  differ  from  those  of  a  2V/2A+  system  only  in  that  video  is  in  the  first  case  switched 
and  in  the  second  continuous.  We  are  not  aware  of  any  existing  2V/2A+  VTT  systems. 


Figure  2.  Hypothetical  arcliitecture  of  2V/2A+  VTT  system. 
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One-Way  Video  with  2-Way  Audio  (1V/2A).  1V/2A  is  widely  used  in  education.The 
architectuie  of  1V/2A  systems  is  illustratexl  in  schematic  form  in  Figure  3.  As  shown  in  Table  1, 
1 V/2A  systems  permit  students  to  see  arid  hear  the  instructor  and  the  instructor  can  hear  but  cannot 
see  students.  The  local  site  has  multiple  cameras  and  can  switch  among  them.  Receiving  sites  do 
not  transmit  video.  As  with  2V/2A  systems,  there  are  generally  one  local  site  and  one  or  more 
remote  sites,  audio  is  continuous,  and  instructor  and  students  participate  in  training  synchronously. 


One-Way  Video  with  1-Way  Audio  (IV/IA).  IV/IA  is  widely  used  in  education.  The 
architecture  of  IV/IA  systems  is  illustrated  in  schematic  fomi  in  Figure  4.  As  shown  in  Table  1, 
1 V/1 A  systems  permit  students  to  see  and  hear  the  instructor  but  the  insuiictor  cannot  see  or  hear 
students.  The  local  site  has  multiple  cameras  and  can  switch  among  them.  Receiving  sites  do  not 
transmit  video  or  audio.  As  with  2V/2A  and  1 V/2A  systems,  there  are  generally  one  local  site  and 
one  or  more  remote  sites,  audio  is  continuous,  and  instructor  and  students  participate  in  training 
synchronously. 

One-Way  Video  with  1-Way  Audio  (IV/IA).  IV/IA  is  widely  used  in  education.  The 
architecture  of  1  V/1  A  systems  is  illustrated  in  schematic  fomi  in  Figure  4.  As  shown  in  Table  1, 
1  V/1  A  systems  permit  students  to  see  and  hear  the  instructor  but  the  instructor  cannot  see  or  hear 
siuderiis.  The  local  site  has  multiple  cameras  and  can  switch  among  them.  Receiving  sites  do  not 
transmit  video  or  audio.  As  with  2V/2A  and  1 V/2A  systems,  there  are  generally  one  local  site  and 
one  or  more  remote  sites,  audio  is  continuous,  and  iasiructor  and  students  participate  in  training 
synchronously. 
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Figure  4.  Hypothetical  architecture  of  IV/IA  VTT  system. 

Viiertape.  Tl-ie  a"-  hite«.cure  of  a  videotape-based  VTT  system  is  illustrated  in  schematic  form 
in  Figure  5.  The  local  site  may  be  identical  to  a  1V/2A  site  (it  may  not  include  a  local  audience), 
out  the  remote  site  consists  of  a  vMeocassette  recorder  and  a  video  display.  As  shown  in  Table  1, 
videotape  permits  stu^enia  to  see  and  boar  the  instructor  but  the  instructor  cannot  see  or  hear 
students.  The  local  site  has  multiple  car.-^ras  and  can  switch  among  them.  Receiving  sites  do  not 
transmit  video  or  audio.  /  s  with  i  V/2A  systems,  there  are  generally  one  local  site  and  one  or  more 
remote  sites  and  audio  is  continuous,  but  instructor  and  students  do  not  participate  in  training 
synchronously.  Note  that  the  only  diffetenc**  between  IV/IA  and  videotape  is  synchiTonicity  of 
participation.  It  is  possible  to  enhance  1 V/1 A  or  videotape  instruction  by  pgividing  an  intemaitient 
audio  link  between  students  and  instructor. 


®  Camera 


Inatructor 


Video  SwKcti 
[Eafol  Camera 


>1 


Ulcrophorea  .^udlo  Ulkar 


Video  Caaaelta  Recorder 


Video  Diaplay 


Video  Caaaatta  Recorder 


OriginatinQ  Site 


Receiving  Site 


Figure  S.  Hypoiheticai  architecture  of  videotape-based  VTT  system. 
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Audiographics.  Audiographics  systems  can  be  built  using  technologies  that  allow  a  local  site 
to  interact  via  2- way  audio  with  a  remote  site  and  to  send  images  to  a  remote  site  using  telephone 
lines.  Two  common  technologies  are  slow-frame  video  and  computer  networks  where  a  central 
computer  communicates  with  remote  computers  to  key  the  display  of  stored  video  images.  The 
architecture  of  a  hypothetical  audiographics  system  is  illustrated  in  schematic  form  in  Figure  6.  As 
shown  in  Table  1,  audiographics  systems  do  not  allow  smdents  or  instructor  to  see  each  other,  but 
graphics  are  displayed  continuously,  the  instructor  can  hear  the  students,  and  instructor  and 
students  participate  in  training  synchronously. 


u 


(ntwcom 


Intercom 


Class 


Originating  Site 


Receiving  Site 


Figure  6.  Hypothetical  architecture  of  audiographics-based  VTT  system. 


Cost  Differences  Among  VTT  Technologies 

Cost  differs  greatly  among  the  VTT  technologies.  Tliose  who  contemplate  the  installation  of  a 
VTT  system  should  consider  costs  carefully  before  implementing  it  and  realize  tliat  many  systems 
fail  for  cost  reasons.  Hershfield  (1986,  July)  surveyed  several  distance  education  systems  that 
failed  and  concluded  tliat  a  common  enor  was  the  tendency  to  focus  attention  on  the  systems  as 
ends  in  themselves  before  planning  how  they  would  be  used.  Ultimately,  large  mnounts  of  money 
were  wasted  and  system  users  became  disillusioned.  The  author  cited  the  Learn  Alaska  Network 
as  an  example  of  this  confusion  and  the  huge  waste  that  can  occur  as  a  result.  In  Learn  Alaska, 
S30,000,0(X)  were  invested  in  building  a  network  that  was  shut  down  a  few  years  later  because  of 
declining  revenues  and  failure  to  meet  its  goals.  Herehfield  stressed  the  importance  of  conducting 
careful,  realistic  cost-benefit  analyses  before  committing  resources  to  system  implementation,  and 
cautioned  that  the  dark  side  of  embracing  high  technology  solutions  is  high  cost  that  may  ultimate¬ 
ly  doom  the  system.  Implicit  in  this  argument  is  that  risk  is  reduced  as  tlie  cost  of  the  VTT  tech¬ 
nology  used  is  reduced,  as  long  as  low  cost  systems  can  satisfy  the  instruciional/conuwunications 
requirements. 
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It  is  difficult  to  obtain  comparative  cost  figures  for  alternative  VIT  technologies.  Published 
estimates  are  based  on  a  host  of  assumptions  and  are  usually  dated.  Cost  changes  as  technology 
evolves,  so  estimates  more  than  a  year  old  may  be  inaccurate.  Moreover,  the  cost  of  a  particular 
VTT  technology  is  governed  by  the  specific  details  of  its  implementation  (e.g.,  how  many 
classrooms,  their  distance  from  one  another,  the  type  of  communication  links  and  VTT  hardware 
used,  hours  of  operation  per  year,  etc.).  Still,  it  is  necessary  to  come  to  terms  with  the  cost  issue  and 
even  a  crude  analysis  reveals  that  there  are  enormous  cost  differences  among  technologies.  TV- 
based  technologies  (2V/2A+,  2V/2A,  1V/2A,  IV/IA)  are  more  expensive  than  videotape  or 
audiographics  by  orders  of  magnitude.  Of  the  TV-based  technologies,  the  capital  cost  of  2V/2A+ 
and  2V/2A  are  close  but  communication  costs  (e.g.,  satellite  transponder  lease  costs)  are  greater 
for  2V/2A+  in  proportion  to  the  number  of  extra  channels.  1 V/2A  is  generally  less  costly  than  2V/ 
2A  because  receiving  sites  do  not  transmit  video  but  only  audio  (which  requires  much  less 
bandwidth),  reducing  communication  cost  greatly.  IV/IA  is  less  costly  than  1V/2A  because 
receiving  sites  do  not  transmit  any  video  or  audio  information  back  to  the  originating  site,  and, 
therefore,  do  nOt  require  equipment  to  transmit  (e.g.,  audio  system,  phone  lines,  satellite  uplink)  or 
incur  communication  costs.  In  gross  cost  terms,  the  technologies  tend  to  shred  out  into  high  cost 
(2V/2A+,  2V/2A),  moderate  cost  (1V/2A,  IV/IA),  and  low  cost  (videotape,  audiographics) 
alternatives,  with  roughly  an  order  of  magnitude  cost  difference  between  levels. 

Differences  Among  Technologies  and  Instruction 

The  distance  education  technologies  described  differ  in  terms  of  the  dimensions  described 
earlier  (see  Table  1)  and  probably  in  other  less  obvious  ways.  These  differences  make  it  reasonable 
to  predict  that  there  will  be  differences  in  student  performance  and  instructor  and  student  attitudes 
depending  upon  the  type  of  technology  used  to  deliver  training.  For  example,  2-way  video  systems 
(2V/2A  and  2V/2A+)  would  appear  to  provide  the  greatest  potential  for  instructor-student 
interaction,  as  instructors  and  students  can  both  see  and  hear  each  other.  One-way  video  systems 
with  audio  links  (1 V/2A)  constrain  interaction,  as  the  instructor  can  hear  but  cannot  see  students. 
Still  more  constraining  are  1-way  video  systems  without  a  return  audio  link  (1 V/1  A)  or  videotape 
instruction,  as  both  eliminate  the  possibility  of  instructor-student  interaction.  Audiographics 
systems  generally  limit  video  displays  to  still  frames,  witli  real-time  annotations,  but  permit  full  2- 
way  audio  interaction. 

Such  a  common  sense  analysis  suggests  that  instructional  effectiveness  will  increase  with 
technological  complexity  and  cost.  Unfortunately,  common  sense  is  not  the  same  as  hard  evidence 
and  cost  does  not  increase  a  little  but  by  orders  of  magnitude  as  technology  shifts  from,  say, 
audiographics  to  1  V/1  A  or  1 V/2A  to  2V/2A.  For  decision  maker’s,  the  main  question  is  whether  the 
utility  of  a  particular  technology  justifies  its  cost.  To  answer  the  question,  one  must  detemiine  (1 ) 
technology  utility  in  tenns  of  objective  metrics  such  as  student  perfonnance  or  user  acceptance  and 
(2)  cost.  Cost  can  be  estimated  by  designing  a  hypothetical  system  and  determining  capital 
invcsmient  and  operating  costs.  Determining  technology  utility  is  more  difficult  because  the 
available  hard  evidence  is  sketchy  and  not  always  trustworthy.  In  any  given  year,  the  professional 
journals  and  trade  magazines  catering  to  distance  educators  and  teleconferencers  publish  a  great 
many  articles  on  new,  experimental,  and  existing  distance  education  systems,  and  almost 
invariably  the  reports  arc  of  successes  in  meeting  educational  or  otlier  staled  goals.  Anecdotal 
repons  and  reports  by  contractors  or  other  parties  with  a  vested  interest  in  system  success  are 
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common.  Reports  of  controlled  research  studies  are  rare.  Also  rare  are  reports  of  system  failures  or 
comparative  studies  of  alternative  distance  education  systems. 

A  comparative  study  of  some  of  the  alternative  VTT  systems  discussed  in  this  report  was 
conducted  by  Beare  (1989).  Beare  compared  six  alternative  distance  education  delivery  methods 
in  a  continuing  education  program.  The  methods  used  were  live  lecture,  lecture  with  videotape 
backup,  TV  lecture  witli  2-way  telephone  hookup  with  microphones  and  a  loudspeaker  at  all  sites 
(equivalent  to  1V/2A),  audio-cassette  assisted  independent  study,  videotape-assisted  independent 
study  (similar  to  IV/IA),  and  videotaped  instruction  with  a  question  and  answer  session  overseen 
by  a  graduate  student  (similar  to  lV/1  A  with  intermittent  audio).  Subsequent  analyses  of  tests  and 
student  course  evaluation  forms  revealed  no  statistically  significant  differences  among  groups, 
although  comments  indicated  that  students  preferred  live  instruction  to  distance  education 
alternatives.  Beare  concluded  that  the  research  demonstrated  that  a  variety  of  inexpensive, 
affordable  alternatives  to  face-to-face  instruction  were  both  effective  and  acceptable  to  students, 
but  cautioned  that  distance  education  was  not  received  favorably  by  students  who  had  a  clear 
option  for  face-to-face  instmction. 

The  finding  of  no  significant  difference  among  technologies,  as  in  Beare’ s  research,  is  common 
in  comparative  studies  and  has  led  some  observers  to  take  the  philosophical  position  that  medium 
does  not  really  matter.  Clark  and  Sugrue  (1990)  recently  conducted  a  meta-analysis  of  several 
different  reviews  of  the  research  relating  to  the  impact  of  medium  on  learning  and  concluded  that 
no  medium  contributed  unique  learning  benefits  that  could  not  be  obtained  from  another  medium. 
Along  the  same  lines,  Nadel  (1988),  noting  that  student  attitudes  did  not  seem  to  be  influenced  by 
the  type  of  distance  education  techn.ology,  concluded  that  students  learn  from  any  medium,  in 
school  or  out,  whether  they  intend  to  or  not,  providing  the  content  of  the  medium  leads  them  to  pay 
attention. 

Though  few  studies  have  compared  a  family  of  related  technologies,  many  have  compared  two 
or  three.  These,  in  aggregate,  may  enable  one  to  extrapolate  about  the  relative  utility  of  different 
technologies. 

2V/2A  has  been  used  successfully  for  Navy  training  in  the  CESN  and  in  the  west  coast  VTT 
demonstration  project.  In  both  cases,  evaluations  have  shown  student  performance  and  attitudes  to 
be  comparable  at  local  and  remote  sites  (Rupinski  &  Stoloff,  1990;  Simpson,  Pugh,  &  Parchman, 
1990).  No  direct  comparisons  to  live  instruction  were  made  in  either  evaluation;  the  control 
condition  was  a  local  classroom  in  a  VTT  system.  In  both  cases,  the  implicit  assumption  was  that 
performance  in  the  local  classroom  would  be  very  similar  to  that  in  a  live  classroom  and  in  both 
cases  only  small  differences  were  found  in  performance  and  attitudes  between  local  and  remote 
classrooms.  Denton,  Clark,  Rossing,  and  O’Conner  (1984,  April)  collected  student  performance 
and  classroom  behavioral  data  during  2- way  interactive  TV  presentations  by  professors  in  the 
College  of  Medicine  at  Texas  A&M  University.  Thirty-two  presentations,  half  televised  and  half 
live,  by  seven  different  faculty  members  were  observed  over  9  months.  Results  indicated  that 
sinnI^•  instructional  strategics  were  used  by  the  instructors  regardless  of  presentation  mode  and 
Jha  ..adem  achievement  was  comparable  in  live  and  TV  classes. 

Nadel  (1988)  compared  student  performance  and  altitudes  in  live  and  remote  classes  within  the 
1 V/2A  TV  network  at  the  University  of  Southern  Maine  and  found  no  difference  in  achievement  or 
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attitudes  between  live  and  TV  classes.  Nadel  did  find  individual  differences  among  students: 
students  had  more  positive  attitudes  toward  TV-based  instruction  as  a  function  of  their  age, 
enthusiasm,  and  preference  for  structure,  and  more  negative  attitudes  as  a  function  of  their 
preference  for  collaboration. 

Flatten  and  Barker  (1987,  June)  described  an  informal  evaluation  of  a  graduate  course 
delivered  via  the  TI-IN  network,  which  uses.  1-way  video  and  1-way  audio  but  allows  students  to 
call  in  via  telephone  (equivalent  to  IV/IA  with  intermittent  audio).  Many  students  were  reluctant 
to  call  and  there  were  delays  and  difficulties  in  coordinating  calls  from  different  sites.  Instructors 
believed  that  lessons  given  via  TV  had  to  be  better  organized  than  those  given  live  and  found 
teaching  without  being  able  to  see  students  challenging.  Most  students  felt  that  instructional  TV 
maintained  their  interest  as  well  as  live  instruction  but  preferred  live  instruction. 

Chute,  a  researcher  on  the  staff  of  the  National  Training  Center,  a  subsidiary  of  AT&T,  has 
published  extensively  on  the  use  of  proprietary  AT&T  audiographics-based  teletraining  systems. 
Chute,  Balthazar,  and  Poston  (1988)  summarized  5  years  of  research  and  development  in  the 
application  of  these  systems  in  corporate  and  higher  education.  Studies  comparing  live  training 
with  teletraining  using  2-way  audio  and  an  electronic  conference  board  demonstrated  that  students 
learned  better  with  teletraining  and  generally  reported  a  high  level  of  satisfaction  with  it,  though 
satisfaction  was  governed  by  several  factors,  including  job  relevance,  cost,  video  quality,  degree 
of  interactiveness,  instmctional  appeal,  student  needs  and  expectations,  and  other  factors.  There 
continues  to  be  some  learner  resistance  to  teletrained  instruction.  Knapczyk  (1990, 1991)  reported 
on  the  successful  use  of  audiographics  to  support  in-service  programs  for  teachers  and  other 
professionals. 

These  studies  provide  insight  into  VTT  technology  utility  but  are  clearly  too  limited  to  provide 
definitive  answers.  Moreover,  most  studies  took  place  in  the  academic  or  business  world,  and  it  is 
reasonable  to  expect  that  the  results  might  not  apply  equally  to  Navy  personnel.  The  research 
described  in  this  report  is  intended  to  provide  additional  data  on  technology  utility  to  aid  decision 
making  concerning  the  selection  of  technologies  for  use  in  future  Navy  distance  education  systems. 
The  research  described  is  a  comparative  study  of  live  instruction  and  six  different  simulated  VIT 
technologies  based  on  those  sketched  earlier  in  this  report, 

METHOD 


Overview 

The  method  included  steps  to  define  assumptions  and  simulation  requirements,  develop  the 
research  plan,  prepare  for  VTT  training,  design  and  install  the  VTT  laboratoty,  and  collect  and 
analyze  data.  These  steps  are  described  below. 

Assumptions  and  Simulation  Requirements 

Assumptions 

Our  research  into  VTT  technologies  was  based  on  several  assumptions: 
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1.  VTT  technologies  can  be  simulated  in  a  laboratory  consisting  of  two  separate  classrooms 
using  closed-circuit  TV.  It  would  have  been  prohibitively  expensive  to  conduct  research  with 
actual  VTT  systems.  The  laboratory  simulations  reduced  cost,  provided  greater  control,  and 
increased  observational  opportunities. 

2.  VTT  training  delivery  procedures  can  be  adapted  from  those  of  traditional  live  instruction 
with  minor  modifications  (e.g.,  refined  visual  aids,  slightly  more  structured  questioning  and  testing 
procedures). 

3.  High  production  values  are  not  essential. 

4.  Students  are  present  in  both  local  and  remote  classrooms. 

5.  Simulations  minimize  personnel  requirements.  The  cost  of  distance  education  systems 
increases  with  number  of  training,  operating,  and  support  personnel.  Simulations  use  one  instructor 
(local  classroom),  one  facilitator  who  is  not  a  subject-matter  expert  (remote  classroom),  and  an  on- 
call  technical  support  person  (local  and  remote  classrooms). 

6.  Class  size  is  approximately  20  students  per  classroom. 

7.  VTT  simulation  does  not  require  compressed-bandwidth  TV.  Many  of  the  technologies  to 
be  simulated  would,  if  implemented,  use  compressed-bandwidth  TV  with  codecs  (coder-decoder) 
at  transmitting  and  receiving  ends.  We  considered  using  codecs  in  the  laboratory  but  rejected  the 
idea  because  codecs  would  add  unnecessary  complexity  and  expense.  The  research  did  not  require 
codecs  because  the  independent  variable  to  be  investigated  (type  of  VTT  technology)  was 
orthogonal  to  the  video  dimensions  a  codec  would  affect  (e.g.,  video  quality,  frame  rate,  and  slight 
time  delays).^ 

Simulation  Requirements 

The  VTT  laboratory  had  to  simulate  the  VTT  technologies  listed  in  Table  1,  allow  easy,  rapid 
changes  among  simulated  technologies,  and  provide  additional  capabilities  for  possible  future 
growth.  The  following  requirements  were  specified: 

Communication  links:  Video  (four  channels),  audio  (two  channels),  intercora/facsimile  (three 

channels). 

Classroom  video:  Up  to  four  cameras  (switchable)  with  large-screen  TV  displays  for  class  and 

small-screen  TV  monitors  for  instructor  and  researchers. 

Classroom  audio:  Audio  mixer  with  sound-activated  microphones  for  instructor  and  up  to  30 

students  and  matching  public  address  system. 


^Othcr  arguincnis  against  using  compressed-bandwidth  TV  arc  ( 1 )  the  level  of  video  quality  for  a  given  bandwidth 
is  constantly  increasing  as  algorithms  arc  improved,  (2)  there  is  no  recognized  way  to  quantify  video  quality,  and  (3) 
the  quality  of  compresscd-bandwidtli  TV  is  converging  toward  conventional  analog  TV.  The  use  of  closed-circuit  rath¬ 
er  than  compiesscd-bandwidih  TV  has  implications  for  inicipreiation  of  the  results,  as  discussed  in  tlic  Results  section. 
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Intercom:  One  telephone  handset 

Facsimile:  One  fasimile. 

Instructor’s  workstation:  Lectern,  table,  easel  camera,  and  video  switch. 

Simulation  requirements  were  identical  in  local  aiid  remote  classrooms.  All  equipment 
identified  above  except  communication  links  (which  were  shared)  had  to  be  duplicated  in  both 
classrooms. 

Research  Plan 

The  research  plan  is  described  below  in  terms  of  its  research  objective,  research  design, 
dependent  variables,  data  collection  methods  and  insiiruments,  subjects,  and  data  collection. 

Research  Objective 

The  research  objective  was  to  investigate  the  relative  training  effectiveness  and  acceptance  by 
students  and  instructors  of  the  type  of  VTT  technology  used.  The  baseline  for  comparison  was 
traditional  live  instruction.  It  was  not  expected  that  any  VTT  technology  would  improve  training 
effectiveness  or  acceptance;  parity  with  live  instruction  would  validate  the  technology. 

Research  Design 

The  research  design  is  illustrated  in  Table  2.  The  independent  variable  was  type  of  VTT 
technology.  Seven  different  research  conditions,  each  corresponding  to  a  separate  technology, 
were  simulated  in  the  laboratory.  In  condition  1  Give  instruction),  the  instructor  taught  a  single  live 
class  in  the  VTT  laboratory.  In  conditions  2-7,  the  instructor  taught  a  live  class  in  the  local 
classroom  and  a  remote  class  via  a  simulated  VTT  technology. 

Table  2 

Research  Design 


Research  Condition 

Local  Classroom 

Remote  Classroom 

1 

Live 

N/A 

2 

Live 

2V/2A+ 

3 

Live 

2V/2A 

4 

Live 

1V/2A 

5 

Live 

IV/IA 

6 

Live 

lV/iA+ 

7 

Live 

Audiographics 
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Research  conditions  2-7  coirespond  to  VTT  technologies  listed  in  Table  1  with  two  exceptions. 
First,  the  research  design  does  not  include  a  condition  corresponding  to  videotape.  The  attributes 
of  videotape  are  essentially  identical  to  W/IA  (see  Table  1);  only  the  IV/IA  condition  was 
included  in  the  design.  Second,  the  design  includes  an  enhanced  IV/IA  condition  (condition  6, 
referred  to  as  1 V/1 A+),  which  allows  intermittent  2- way  audio  communication  in  what  is  primarily 
a  1  V/1  A  simulation;  this  communication  occurred  at  six  specific  times  during  the  instructional  day 
for  an  elapsed  time  of  approximately  15  minutes  each. 

Audiographics  was  simulated  by  combining  2- way  audio  with  a  1-way  TV  broadcast  of  an 
easel  camera  display.  Note  that  this  simulation  differs  from  audiographics  technologies  which 
display  computer-based  images. 

Dependent  Variables 

Dependent  variables  were  student  performance  on  written  examinations  (daily  quizzes  and 
final)  and  student  attitudes  on  several  different  factors  as  reflected  in  written  course  evaluations. 
In  addition,  data  were  collected  concerning  classroom  processes,  classroom  observation  log 
entries,  and  instructor  debriefings.  All  forms  of  data  were  collected  for  all  research  conditions.  The 
methods  and  instruments  used  to  collect  these  data  are  described  below. 

Data  Collection  Methods  and  Instruments 

Student  background  questionnaire:  Students  completed  a  one-page  “Student  Survey” 
(Appendix  A)  at  the  start  of  each  course.  The  questionnaire  provides  information  on  student 
seniority  and  course  subject-matter  experience. 

Final  examinations:  Written  final  examinations  were  administered  to  students  at  the  end  of  the 
course.  The  exams  were  taken  “open  book"  and  consisted  of  50  fill-in  items.  There  were  two 
equivalent  forms  of  the  final. 

Daily  quizzes:  Written  15-item  quizzes  were  administered  to  students  at  the  beginning  of  class 
on  days  2,  3,  and  4.  The  exams  were  taken  “c^n  book.” 

Student  course  evaluatif>ns:  Attitude  measures  were  obtained  using  Likert  rating  scales  on  a 
series  of  questions  relating  to  die  insttuctor,  audio-visual  aids,  tests/homework,  overall  assessment, 
and  instructor-student  interaction.  The  questionnaire  included  a  series  of  semantic  differential 
items  relating  to  course  content  and  fonn  of  instruction.  Student  comments  on  the  class  were 
gathered  in  a  series  of  open-ended  questions.  A  sample  evaluation  questionnaire  is  contained  in 
Appendix  B. 

Observations:  Navy  Personnel  Research  and  Development  Center  (NPROC)  observers  were 
present  in  the  receiving  classroom  on  days  1,  2,  and  3  of  each  week.  The  observer  sat  in  the  back 
of  the  classroom  at  a  table  containing  two  13"  TV  monitors.  One  monitor  showed  the  view  from 
the  front  of  the  local  classroom  and  the  other  the  view  from  the  front  of  tlie  remote  classroom.  From 
this  vantage  point,  the  observer  could  see  all  students  in  both  classes  on  TV  and  observe  the  remote 
class  directly.  The  observer  maintained  a  written  log  describing  significant  events  in  the  class  as 
they  occurred  (e.g.,  instructor-student  interactions,  training  activities,  problems,  technical 
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difficulties).  In  addition,  twice  during  days  2  and  3  of  each  week  the  observer  filled  out  a 
“Classroom  Interaction  Analysis  Form”  to  record  the  classroom  processes  in  a  structured  way.  A 
sample  form  is  contained  in  Appendix  C. 

Instructor  debriefings:  Instructors  were  debriefed  by  NPRDC  researchers  each  week  on  the 
morning  following  the  final  day  of  the  class.  By  this  time  instructors  had  graded  tests,  reviewed 
student  questionnaires,  and  reflected  on  their  classroom  experiences  during  the  week.  Instructor 
comments  were  recorded  in  the  classroom  log. 

Subjects 

Subjects  were  Navy  active  duty  and  reservist  personnel  undergoing  training  required  by  their 
duty  position.  Students  were  assigned  to  a  classroom  (local  or  remote)  by  the  3M  instructional 
staff,  who  attempted  to  balance  the  relative  sizes  and  seniority  of  local  and  remote  classes.  Each 
classroom  typically  contained  about  20  students.  Average  class  sizes  for  local  and  remote 
classrooms  as  a  function  of  research  condition  and  classroom  are  given  in  Table  D-1.^ 

Seniority  (rank)  was  assumed  to  reflect  3M  knowledge  and  experience  and  a  seniority  index 
was  computed  for  each  class  by  assigning  a  number  to  each  student  based  on  rank  and  computing 
the  average  for  the  class.  The  ranking  scheme  was  E-1  through  E-4  (1),  E-5  (2),  E-6  (3),  E-7 
through  0-2/W0-1  through  WO-4  (4),  0-3  through  0-5  (5).  For  example,  a  student  with  the  rank 
of  E-4  would  be  assigned  the  number  1,  an  0-2  would  be  assigned  a  4,  and  so  forth.  The  assumption 
and  ranking  scheme  enable  comparison  of  seniority  level  as  a  function  of  research  condition  and 
classroom  (Table  D-2).  Seniority  varied  across  research  conditions  and  variance  due  to  this  factor 
was  handled  statistically  during  data  analysis  (see  Results). 

Data  Collection 

Data  were  collected  over  a  6-month  period  with  two  breaks.  Out  strategy  was  to  collect  data 
for  a  particular  research  condition  during  3  successive  weeks,  reconfigure  the  laboratory  as 
necessary  to  simulate  the  next  condition,  collect  data  for  that  condition  for  3  weeks,  and  so  forth. 
The  order  in  which  data  for  the  different  research  conditions  was  collected  was  1, 3, 4, 2, 6, 7, 5. 
The  extended  3-week  data  collection  periods  enabled  instructors  to  become  familiar  and 
comfortable  with  the  particular  technology  simulation,  provided  a  large  subject  pool  for  statistical 
analyses,  and  balanced  out  some  of  tlie  extraneous  factors  beyond  our  control  (see  below). 

The  research  was  coiducicd  wiiiiin  a  Navy  schoolhousc,  which  meant  that  our  njsearch  could 
not  impose  training  conaiiions  on  students  that  might  irfcversibly  compromise  their  learning.  To 
avoid  possible  problems  in  the  more  austere  research  conditions  (5, 6,  and  7),  a  decision  was  made 
to  impose  each  research  treatment  during  only  the  second  day  of  the  4-day  course  rather  tlian  for 
the  entire  4  days.  We  were  thus  able  to  assess  student  test  perfonnance  on  course  content  covered 
during  day  2  wdth  a  quiz  on  the  day  following  the  research  treatment,  which  would  have  enabled 
remediation  if  student  learning  had  suffered.  U  would  have  been  preferable  to  impose  the  research 
treatment  for  the  entire  course  but  this  was  not  Icasiblc  within  tlic  schoolhousc  environment. 


^Because  of  the  large  nmnber  of  tables  in  this  report,  only  tables  of  primary  interest  arc  presented  in  the  body  of 
the  text.  Tables  of  secondary  interest  arc  in  Appendix  D. 
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Six  different  instructors  and  three  different  facilitators  participated  in  the  research.  Personnel 
assignments  were  governed  by  availability,  competing  instructional  demands,  and  instructor 
turnover.  It  would  have  been  desirable  but  was  not  possible  to  balance  instructor  and  facilitator 
assignments  in  a  systematic  way  across  conditions.  E)espite  this  constraint,  we  have  no  reason  to 
suspect  that  the  way  in  which  these  assignments  were  made  affected  the  dependent  variables. 

Preparation  for  VTT  Training 

Training  Course  Selection 

During  previous  research,  we  developed  criteria  and  conducted  training  process  analyses  to 
identify  and  select  courses  to  use  in  distance  education  research  (Simpson,  l^gh,  &  Parchman, 
1990).  Selection  criteria  were  frequent  convening,  significant  class  size,  no  need  for  special 
equipment,  and  length  of  less  than  1  week.  We  observed  several  Navy  training  courses,  conducted 
a  training  analysis,  and  determined  that  the  Supervisor’s  3M  Administration  and  Operations  course 
would  be  a  good  candidate.  3M  is  a  high-backlog,  4-day,  lecture-based  maintenance  record¬ 
keeping  course.  It  requires  presentation  of  graphics,  which  must  be  annotated;  has  laboratories 
during  which  students  fill  out  written  forms,  which  must  be  reviewed  by  the  instructor,  who 
provides  individual  feedback  to  students;  and  requires  written  tests  to  be  administered.  Live  3M 
courses  invariably  call  for  considerable  instmctor-student  interaction  in  the  form  of  questions, 
answers,  discussion,  and  miscellaneous  administrative  matters.  The  3M  course  was  selected 
because  it  (1)  has  a  steady  flow  of  students  and  would  facilitate  data  collection  and  (2)  is  rich  in 
terms  of  the  training  processes  it  requires  and  research  results  based  on  it  generalize  to  other 
lecture-based  Navy  training  courses. 

Classroom  Procedures 

Classroom  procedures  were  constrained  by  the  audioA^ideo  equipment  used,  classroom  design 
(both  described  later),  and  the  requirement  to  conduct  training  and  manage  two  classes 
simultaneously.  The  classroom  had  a  stationary  camera  pointed  at  the  instructor  (who  stood  behind 
a  lectem)  and  a  second  stationary  camera,  pointed  at  a  white  board  to  the  instructor’s  left,  which 
could  be  switched  on  to  allow  the  instructor  to  write  on  the  white  board,  but  which  was  never  used 
(instructors  preferred  to  write  instead  on  paper  on  the  bascboanl  of  an  easel  camera).  Instructors 
had  to  restrict  their  movement  left  and  right  to  remain  in  the  picture  frame.  The  easel  camera  w'as 
an  effective  alternative  to  a  while  board  and  proved  superior  in  its  ability  to  zoom,  prcscjit  color, 
and  show  ilinec-dimcnsional  objects. 

No  modifications  were  made  to  the  contatt  of  lectures,  classtoom  exercises,  tests,  or  other 
classroom  materials.  VTT  and  live  course  length  were  identical.  Viewgraphs  us«]  in  the  live  class 
were  cleaned  up  and  convened  to  hard  copy  fomi  to  improve  ilicir  appearance  on  TV  displays. 

Traditional  insinicior-studcnt  interaction  procedures  were  modified  for  VTT'.  The  instructor 
made  scrting  chans  of  both  classrooms  and  systematically  alternated  questions  between 
classrooms.  In  some  research  conditions,  the  instructor  could  see  students  in  the  remote  classroom 
on  a  TV  motuior  but  not  usually  well  enough  to  identify  who  was  speaking.  In  all  research 
conditions,  students  were  required  to  identify  self  by  name,  pause  to  be  recognized  by  the 
instructor,  and  then  asked  their  question.  The  instructor  w  ould  sontciimcs  repeat  a  question  before 
answering,  hi  some  conditions,  students  could  not  speak  to  the  instructor. 
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Classes  included  laboratories  during  which  students  filled  out  written  forms  and  received 
individual  feedback  on  their  answers  from  the  instructor.  In  a  live  class,  the  instructor  would  stroll 
the  classroom  to  provide  help.  This  was  impossible  in  the  remote  VTT  class  so  the  instructor  had 
remote  students  send  their  work  to  him  via  camera,  facsimile  machine,  or  both  and  would  talk  to 
students  individually  using  a  telephone  handset.  In  research  conditions  that  did  not  allow  remote 
students  to  speak  to  the  instmctor,  they  did  not  receive  individual  help;  the  instmctor  worked 
individually  with  students  in  the  local  classroom  but  only  reviewed  the  correct  answers  on  work 
sheets  for  the  remote  class. 

Multiple-choice  paper  and  pencil  tests  were  administered  and  scored  in  the  local  classroom  by 
the  instructor;  this  was  done  in  the  renriote  classroom  by  the  facilitator,  and  the  results  were 
transmitted  back  to  tlie  instructor  in  the  originating  classroom  using  the  facsimile  machine. 

Instructor  Training 

Six  different  instructors  delivered  training  in  the  VTT  laboratory.  All  instructors  had  graduated 
from  Navy  instructor  training  school  and  were  qualified  to  teach  3M.  NPRDC  research  personnel 
worked  closely  with  the  first  two  instructors  during  an  informal  1-week  training  period.  Instructors 
were  familiarized  with  the  audio  and  video  equipment  and  practiced  equipment  operation  and 
classroom  procedures.  The  total  training  period  per  instructor  was  approximately  2  days. 

The  two  initial  instructors  subsequently  trained  the  other  four  instructors  who  taught  in  the 
VTT  laboratory.  The  training  received  by  later  instructors  went  more  rapidly  than  that  received  by 
the  initial  instructors.  It  came  as  something  of  a  revelation  to  us  that  instructors  could  become 
competent  VTT  instructors  with  about  two  days  of  infonnal  training  and  practice  and  few  days 
teaching  in  a  VTT  classroom.  None  of  the  instructors  was  given  or  had  previously  received  training 
in  camera  presence,  articulation,  graphics  production,  or  oilier  skills  of  TV  professionals. 

\nrT  Laboratory 

The  VTT  laboratory  is  described  below  in  terms  of  equipment  selection  and  classroom  design. 

Cquipinetii  Selection 

TV  camera.s:  Simulation  requirements  called  for  four  different  T'V  cameras  in  each  classroom: 
(1)  instructor,  (2)  easel  camera.  (3)  class,  and  (4)  au.xiliary.  Wc  selected  high-quality  single-chip 
CCD  color  cameras  for  all  applicauon.s.  A  Panasonic  \VV-D5 1  (X)  canrera  with  1 2: 1  zoom  lens  was 
u-sed  for  the  instructor  and  Panasonic  WV-CLl  10  camcm  with  wide-angle  lenses  w'crc  u,scd  for 
class  and  auxiliary  cameras.  The  WV-D5100  is  widely  used  in  audio- visual  work  and  the  WV- 
Q..1 10  mainly  in  surveillance  applications.  Elnro  easel  canreras  were  used;  the  Einro  is  a  compact, 
dedicated,  easel  camera  rcscrtibling  an  overhead  projector.  A  multi-channel  video  swatch  was  used 
to  select  which  camera’s  signal  to  send  to  the  other  classroom. 

TV  displays  and  numitor^:  Largc-.sizc  TV  displays  were  required  to  present  pictures  to  the 
class  and  smaller  monitors  to  prc.'Jcm  pictures  to  single  view-ers  (snsirucior  and  researchers).  Most 
.simulations  required  a  single  TV  display,  though  some  txx^uired  two  or  even  three.  There  was 
unccnaimy  about  what  size  display  to  use.  Many  instructional  TV  systems  use  25"  TV  displays, 
often  several  in  a  single  classroonr,  though  larger  displays  seem  to  be  bcctmhng  more  crnirmon. 
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Guidance  on  monitor  size  selection  and  on  other  aspects  of  the  use  of  monitors  in  class  is  limited, 
as  we  noted  in  previous  research  (see  Simpson,  Pugh,  &  Parchman,  1990).  We  obtained  25",  35", 
and  45"  displays  and  had  students  in  two  classes  declare  their  preferences  for  Mitsubishi  35"  tube 
or  45"  rear-projection  displays  after  4  days  of  viewing  them  in  parallel.  Students  preferred  the 
larger  display  by  a  margin  of  about  8:1  and  we  subsequently  used  it  as  the  main  display  in  all 
simulations.  Sony  13"  TV  monitors  were  used  to  present  pictures  to  the  instmetor  and  researchers. 

Audio  system:  We  selected  Shure  AMS-8000  eight-channel  audio  mixers  and  AMS-22  low- 
profile  sound-activated  table  microphones.  The  instructor  wore  a  wired  clip-on  Shure  model  839 
lavalier  microphone;  the  mixer  channel  for  this  microphone  was  modified  so  that  the  microphone 
was  continuously  on  rather  than  sound  activated.  Each  classroom  was  equipped  with  two  audio 
mixers,  whose  combined  output  was  fed  via  audio  cable  to  a  75  watt  public  address  system  in  the 
other  classroom. 

Other  equipment:  Each  classroom  was  equipped  with  a  telephone  connected  to  the  telephone 
in  the  other  classroom  via  ring-down  telephone  circuit.  Picking  up  one  receiver  caused  the  bell  of 
the  other  telephone  to  ring  and  permitted  talk  between  classrooms.  Each  classroom  was  equipped 
with  a  Panasonic  lJF-250  facsimile  machine  with  its  own  line  to  the  other  classroom.  Each 
classroom  was  also  equipped  with  various  other  items  of  equipment,  including  TV  monitor  racks, 
camera  hangers  aiid  mounts,  tables  and  chairs,  etc. 

Classroom  Design 

Both  classrooms  were  equipped  witli  overhead  fluorescent  lights,  which  were  modified  by 
adding  a  dimmer  switch  and  using  tubes  balanced  for  a  color  temperature  of  3200  degrees  Kelvin  to 
assure  that  flesh  tones  would  look  namral  on  camera.  No  special  lighting  was  used  in  classrooms; 
adjustable  window  blinds  controlled  the  entry  of  external  light.  The  front  wall  of  each  classroom 
was  painted  pale  blue  and  cleared  of  clutter  to  provide  a  suitable  backdrop  on  camera.  Both 
classrooms  were  carpeted  to  reduce  echoes  and  reverberation.  Ceilings  had  acoustic  tiles. 

Classroom  sizes  were  30’  by  30’  (local)  and  30’  by  40’  (remote).  Both  rooms  provided  ample 
space  fOi“  arranging  furniture  and  equipment.  Figure  7  shows  the  floor  plan  of  both  classrooms. 
Students  sat  at  72"  X  30"  tables,  with  two  chairs  per  table.  Each  table  was  equipped  witli  a  low- 
profile  microphone.  The  tables  were  tirranged  in  amphitheatre  fashion  so  that  all  students  would  be 
seated  within  the  90-degree  arc  originating  from  the  center  of  the  45"  TV  display;  tliis  assured  that 
all  students  could  view  the  primaiy  TV  display  adequately.  Two  additional  TV  displays  were 
located  to  the  left  of  the  primary  display.  In  research  conditions  providing  2-way  video,  tlie  35"  TV 
in  the  originating  classroom  was  moved  to  the  back  of  the  classroom  so  that  it  could  be  used  to 
display  the  remote  class  to  the  instructor. 

The  instructor’s  primary  camera  was  suspended  above  the  second  row  of  tables.  The  class 
camera  was  located  above  and  behind  the  45"  TV  display.  The  instructor’s  auxiliary  camera  was 
suspended  forward  of  the  primary  camera  and  to  its  right  so  that  it  covered  an  area  that  might  be 
used  for  a  white  board;  the  auxiliary  camera  was  never  used  during  research.  The  in.structor  stood 
behind  a  lectern  at  the  front  of  the  classroom  to  the  left  of  the  primaiy  TV  display.  On  the  table  to 
his  left  were  an  easel  camera  and  two  13"  TV  monitors;  one  monitor  showed  outgoing  video  and 
the  other  monitor  was  either  unused  or  showed  incoming  video  (in  research  conditions  providing 
2-way  video).  The  table  also  held  the  facsimile  machine  and  telephone  handset. 
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Figure  7.  VTT  classroom  floor  plan. 


The  video  switch  (a  small  panel  with  seven  push  buttons)  was  attached  to  tlie  left  side  of  the 
lectern  so  that  the  instructor  could  reach  down  and  switch  cameras  with  a  single  unobtrusive 
motion  of  the  left  hand.  In  most  research  conditions,  tlie  instructor  used  this  switch  to  control  which 
of  the  four  cameras  to  switch  on  and  in  turn  display  on  tlie  45"  TV  in  tlie  originating  classroom  and 
transmit  to  the  remote  classroom. 


RESULTS 


Overview 

Research  conditions  1 ,  2.  3,  and  4  occurred  Uiroughout  die  4-day  course,  but  conditions  5,  6, 
and  7  occurred  only  during  day  2.  Many  of  Uie  data  collection  instruments  reflect  the  cumulative 
effect  of  die  entire  course,  and,  therefore,  could  not  be  used  in  detcmiining  the  effects  of  conditions 
5,  6,  and  7,  Comparisons  of  the  cffecis  of  conditions  1,  2,  3,  and  4  could  be  made  using  all  forms 
or  data;  the  first  subsection  makes  these  comparisons.  The  second  subsection  compares  conditions 
1,  5,  6,  and  7  using  die  appropriate  subset  of  data.  No  direct  comparisons  are  made  among 
conditions  2,  3,  and  4  and  conditions  5, 6,  and  7,  though  inferences  may  be  drawn  by  reference  to 
condition  1,  which  1$  the  common  denominator  in  other  cotnpari.sons. 
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As  noted  in  the  Method  section,  the  study  used  conventional  rather  than  compressed-bandwidth 
TV.  If  the  experiment  were  replicated  with  compressed-bandwidth  TV,  any  differences  between 
live  and  VTT  conditions  might  increase  slightly,  but  no  differences  among  VTT  conditions  would 
be  expected. 

Comparisons  Among  Conditions  1, 2, 3,  and  4 

Conditions  are  compared  below  based  on  (1)  Hnal  examination,  (2)  course  questionnaire,  and 
(3)  observations. 

Final  Examination 

Table  D-3  shows  the  raw  final  examination  scores  for  all  subjects  in  conditions  1,  2, 3,  and  4 
in  local  and  remote  classrooms.  The  data  are  summarized  in  bar  graph  form  in  Figure  8. 


100 


Live  2V/2A+  2V/2A  1V/2A 

Conditioo 


Figure  8.  Bar  graph  of  raw  final  examination  scores 
for  conditions  1, 2, 3, 4. 


Student  seniority  varied  with  condition  (sec  Table  D-2)  and  might  influence  perfomiance  on 
the  final  examination  and  mask  the  effects  of  the  independent  variable.  The  data  were  analyzed 
using  an  hierarchical  analysis  of  covariance  (Cohen  &  Cohen,  1975;  Overall  &  Spiegel,  1969). 
Seniority,  the  covariate,  was  found  to  have  a  significant  effect  on  final  examination  performance, 
F(  1 ,4 1 8)  =  34.64,  p  <  .01 .  Variance  due  to  the  covariaic  was  removed  in  subsequent  comparisons, 
which  were  made  in  Uie  following  order 

1 .  Live  (condition  1 )  versus  All  VTT  (c'onditions  2, 3, 4). 

2.  1 V/2A  (condition  4)  versus  2V/2A  and  2V/2A+  (conditions  2,  3) 
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3.  2V/2A  (condition  3)  versus  2V/2A+  (condition  2). 

4.  Local  classroom  (across  conditions  2, 3, 4)  versus  remote  classroom  (across  conditions  2, 
3,4). 

Table  3  summarizes  the  results  of  this  analysis.  The  amount  of  variance  accounted  for  by 
seniority  was  statistically  significant  and  was  by  far  the  largest  effect  shown  in  Table  3. 

Table  3 

Hierarchical  ANCOVA  Results  for  Conditions  1, 2, 3, 4 


Comparison 

Variable  1 

Mean  1 

Variable  2 

Mean  2 

df 

F 

P 

Live  vs.  all  VTT 

(2,3.4) 

Live 

90.03 

All  VTT  (2. 3.4) 

88.02 

1,418 

3.93 

<.05 

2V/2A  &  2V/2A+ 

vs.  1V/2A 

2V/2A  &  2V/2A+ 

87.44 

1V/2A 

89.08 

1,418 

4.01 

<.05 

2V/2A  vs.  2V/2A+ 

2V/2A 

87.49 

2V/2A+ 

87.40 

1.418 

<1 

NS 

Local  vs.  Remote 

Local 

88.56 

Remote 

87.47 

1.418 

2.56 

NS 

The  difference  between  Live  and  All  VTT  was  small  but  statistically  significant.  Students  in 
live  classes  scored  about  2  points  higher  than  VTT  students  but  VTT  should  not  be  expected  to 
degrade  student  performance  in  any  meaningful  way.  This  finding  provides  a  stronger  foundation 
for  comparisons  between  live  and  VTT  instruction  than  had  existed.  In  previous  research,  we  had 
been  unable  to  use  a  live  class  as  a  control  condition  in  assessing  VTT,  but  had  used  instead  the 
originating  classroom  in  a  VTT  system  (Simpson,  Pugh,  &  Parchman,  1990)  and  found  no 
significant  performance  difference  between  classes.  However,  a  live  class  is  not  tmly  equivalent  to 
an  originating  VTT  class  (e.g.,  class  sizes  differ,  tlie  mstructor  has  to  attend  to  video  equipment, 
the  remote  class  must  be  monitored,  etc.). 

The  difference  between  1-way  video  and  2- way  video  was  small  but  statistically  significant. 
Students  in  I -way  video  classes  scored  about  1.5  points  higher  than  students  in  2- way  video 
classes.  Tliis  finding  is  counter-intuitive,  as  it  would  seem  that  2-way  video  is  a  richer  environment 
than  1-way  video.  There  is  no  difference  between  these  conditions  from  the  students’  point  of  view 
but  there  is  from  tlie  instructor’s.  With  2-way  video,  the  instructor  can  see  the  remote  class  but  with 
1  -way  video  he  or  she  cannot.  A  possible  explanation  for  the  difference  is  that  2-way  video  imposes 
additional  burdens  on  the  instructor,  who  mast  (1)  allocate  attention  to  the  TV  showing  the  remote 
class,  (2)  rely  on  visual  cues  from  tlie  remote  class  (e.g.,  raised  hand)  in  a  low-grade  visual 
environment  rather  than  on  auditory  cues  to  attract  the  instructor’s  attention,  and  (3)  syntliesize 
auditor)'  and  visual  cues  that  arc  contradictory  (location  of  the  voice  from  the  speaker  is  not  in 
agreement  with  student  image  on  the  monitor).  One-way  video  relieves  tlie  instructor  of  tliesc 
burdens,  although  it  renders  him  or  her  blind  to  the  remote  class. 

The  difference  between  single-channel  2-way  video  and  multiple-channel  2-way  video  was 
not  statistically  significant. 
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The  difference  between  local  classroom  and  remote  classroom  was  not  statistically  significant. 
This  finding  is  consistent  with  our  earlier  research  (Simpson,  Pugh,  &  Parchman,  1990). 

Course  Questionnaire 

Student  attitudes  were  measured  with  a  post-course  questionnaire  (Appendix  B),  which 
contained  a  series  of  statements  to  be  rated,  multiple-choice  questions,  and  open-ended  questions. 
The  questionnaire  was  administered  to  all  students  taking  courses  at  live,  local  VTT,  and  remote 
VTT  classrooms. 

Student  Ratings.  The  statements  to  be  rated  fell  into  six  categories  (instructor,  audiovisual 
aids,  tests  and  homework,  overall  assessment  of  instructor  and  course,  course  content,  form  of 
instruction).  Questions  1-21  were  rated  on  a  5-point  scale  with  a  midpoint  of  3.  Questions  30-39 
were  rated  on  a  7-point  scale  with  a  midpoint  of  4.  Mean  ratings  were  computed  for  local  and 
remote  classrooms  in  conditions  1-4.  The  majority  of  ratings  on  all  items  fell  well  above  the 
midpoint  on  the  rating  scale;  most  students  gave  positive  ratings  to  the  dimension  being  measured. 
To  assess  statistical  significance,  each  condition,  local  and  remote,  was  compared  with  condition 
1,  live  instruction,  using  a  post  hoc  analysis  of  variance.  Six  comparisons  were  made  for  each  of 
the  31  questions.  No  statistically  significant  differences  were  found  in  any  of  the  comparisons. 
Only  minor  patterns  were  present  in  the  data,  as  discussed  below. 

Table  D-4  shows  student  ratings  on  statements  relating  to  the  instructor.  Whether  taking  the 
course  live  or  in  one  of  the  VTT  conditions,  students  perceived  instructor  performance  to  be 
outstanding. 

Table  D-5  shows  student  ratings  on  statements  relating  to  audiovisual  aids.  Ratings  were  high 
on  all  questions  relating  to  video  and  graphics,  but  in  2V/2A  (condition  3),  remote,  students  gave 
lower  ratings  on  questions  10  and  1 1,  which  relate  to  audio  transmission  loudness  and  clarity.  In 
reviewing  other  data  associated  with  this  phase  of  data  collection,  we  have  reached  the  conclusion 
that  tiiese  lower  ratings  reflect  the  unique  audio  problems  experienced  during  early  data  collection 
and  are  not  a  general  property  of  2V/2A. 

Table  D-6  shows  student  ratings  on  statements  relating  to  tests  and  homework.  Ratings  on 
question  15  were  somewhat  lower  than  on  other  questions,  regardless  of  condition,  but  no  other 
patients  are  apparent  in  the  data. 

Table  D-7  shows  ratings  on  statements  relating  to  overall  assessment  of  instructor  and  course. 
There  are  no  patterns  of  interest. 

Table  D-8  shows  ratings  on  semantic  differential  items  relating  to  course  content.  There 
appears  to  be  a  pattern  throughout  the  data  of  students  in  VTT  local  classrooms  giving  slightly 
higher  ratings  titan  students  in  corresponding  remote  classrooms,  tliough  differences  are  small  and 
not  statistically  significant  and  all  ratings  were  high. 

Table  D-9  shows  ratings  on  semantic  differential  items  relating  to  form  of  instruction.  There 
appears  to  be  a  pattern  throughout  the  data  of  students  in  VTT  local  classrooms  giving  slightly 
higher  ratings  than  students  in  corresponding  remote  classrooms,  though  differences  are  small  and 
not  siati.stically  significant  and  all  ratings  are  high. 
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Table  D>10  shows  student  responses  to  question  22.  The  data  are  collapsed  across  VTT 
conditions  and  summarized  graphically  in  Figure  9.  Differences  among  group  frequencies  were 
analyzed  using  a  chi-square  test  for  goodness  of  fit,  and  differences  among  the  three  groups  were 
found  to  be  significant,  (2  df)  =  10.53,  p  <  .01.  The  percentage  of  students  who  responded  “yes” 
was  highest  in  the  live  class,  lower  in  VTT  local  classes,  and  lowest  in  VTT  remote  classes. 
Overall,  however,  the  majority  of  students  responded  positively  to  the  question. 

100^ _ _ 


Condition 


Figure  9.  Student  responses  to  question  22  (Did  you  talk 
to  the  instructor  or  ask  any  questions  during  the 
regular  hours  of  this  course?),  collapsed  across 
VTT  conditions. 

Table  D-11  shows  student  responses  to  question  23.  The  data  are  collapsed  across  VTT 
conditions  and  summarized  graphically  in  Figure  10.  Differences  among  group  frequencies  were 
analyzed  using  a  chi-square  test  for  goodness  of  fit,  and  differences  among  the  three  groups  were 
found  to  be  significant,  (4  df)  =  26.87,  p  <  .001.  Most  students  felt  either  that  VTT  had  no  effect 
on  opportunities  to  ask  questions  or  provided  more  opportunities.  However,  in  remote  classes, 
overall,  17.7  percent  of  students  felt  that  VTT  had  provided  fewer  opportunities. 

Table  D-12  shows  student  responses  to  question  28.  The  data  are  collapsed  across  VTT 
conditions  and  summarized  graphically  in  Figure  11.  Differences  antong  group  frequencies  were 
analyzed  using  a  chi-square  test  for  goodness  of  fit,  and  differences  among  the  three  groups  were 
found  to  be  significant,  (4  df)  =  17.96,  p  <  .01.  Approximately  30  percent  of  students  in  local  and 
remote  classes  expressed  preference  for  traditional  instruction;  the  majority  eitlier  expressed  no 
preference  or  preferred  VTT.  Interestingly,  students  in  the  remote  class  were  about  twice  as  likely 
to  prefer  VTT  as  students  in  the  local  class. 
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Figure  10.  Student  responses  to  question  23  (How  did  the  VTT 
method  of  instruction  affect  your  opportunities  to 
talk  to  the  instructor  or  ask  questions,  as  compared 
to  traditional  methods  of  instruction?). 


VTT.Local 

Condition 


VTT-Remote 


□  VTT 
^  Traditional 
H  No  Preference 


Figure  11.  Student  responses  to  question  28  (Which  method 
of  instruction  would  you  have  preferred  for  this 
course?). 


23 


Table  D-13  shows  student  responses  to  question  29.  The  data  are  collapsed  across  VTT 
conditions  and  summarized  graphically  in  Figure  12.  Most  students  either  felt  that  VTT  had  no 
effect  on  learning  or  improved  learning.  A  chi-square  goodness  of  fit  analysis  did  not  reveal  any 
significant  differences  in  frequency  of  choice  of  responses  among  the  three  groups  of  respondents. 


H]  Improved  learning 
H  No  effect 
B  Reduced  learning 


VTT-Local  VTT-Remote 

Condition 

Figure  12.  Student  responses  to  question  29  (How  did 
the  participation  of  student  at  other  site(s) 
affect  your  learning  during  this  course?). 

Student  Comments.  Comments  were  solicited  from  the  429  students  in  conditions  1, 2, 3,  and 
4  with  three  open-ended  questions  (40,  41,  42)  relating  to  student  likes  and  dislikes  about  the 
course  and  suggestions  for  improvements.  The  continents  were  exhaustively  listed  and  then 
categorized  for  live  instruction  (condition  1)  and  VTT  (conditions  2,  3,  4)  in  local  and  remote 
classrooms. 

The  breakdown  of  comments  for  question  40  (student  likes)  is  shown  in  Table  4.  The  three 
most  common  comments  related  to  course  content,  instructor,  and  audio-visual.  The  pattern  of 
comments  is  similar  for  the  three  listing  categories  with  two  exceptions:  (1)  live  and  VTT  local 
students  were  more  likely  than  VTT  remote  students  to  comment  on  the  instructor  and  (2)  VTT 
remote  students  were  more  likely  to  comment  on  audio-visual  than  live  or  VTT  local  students. 

The  breakdown  of  comments  for  question  41  (student  dislikes)  is  shown  in  Table  5.  The  four 
most  common  comments  related  to  dry  material,  audio-visual,  sound  problems,  and  tests/quizzes/ 
homework.  The  pattern  of  comments  is  similar  for  the  three  listing  categories  with  two  exceptions: 
(1)  live  and  VTT  local  students  were  more  likely  than  VTT  remote  students  to  comment  on  dry 
material  and  (2)  VTT'  remote  students  were  more  likely  to  comment  on  sound  p’oWems  than  live 
or  VTT  local  students. 
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Table  4 


Student  Responses  to  Question  40: 
What  did  you  like  most  about  this  course? 
(Numbers  are  percentages.) 


Comment 

Live 
(N  =  40) 

VTT 

Local 
(N  =  158) 

VTT 
Remote 
(N  =  142) 

Course  content 

45 

44 

45 

Instructor 

30 

26 

15 

Audio-visual 

5 

11 

22 

Exercises/using  manual 

10 

7 

8 

Classroom  environment 

3 

6 

1 

Usage  of  time 

5 

4 

4 

Working  with  other  students 

3 

1 

6 

The  breakdown  of  comments  for  question  42  (student  suggestions)  is  shown  in  Table  6.  Note 
that  (1)  the  majority  of  comments  reflect  VTT  audio  or  video  and  (2)  remote  students  made  many 
more  suggestions  than  local  students.  Remote  students  were  probably  more  aware  of  VTT  than 
local  students  because  VTT  deficiencies  were  more  likely  to  affect  them.  The  pattern  of  responses 
is  similar  for  local  and  remote  classrooms  with  two  exceptions:  (1)  remote  students  were  more 
likely  than  local  students  to  suggest  improving  audio  and  (2)  local  students  were  more  likely  than 
remote  students  to  express  a  desire  to  see  the  other  class.  Many  students  in  both  classrooms 
suggested  that  push-to-talk  microphones  should  be  used.  The  motivation  appeared  to  be  either  that 
sound-activated  microphones  (1)  resulted  in  unwanted  sounds  being  generated  (for  example, 
fingers  tapping  on  tables)  and  interfering  with  student  understanding  or  (2)  precluded  private 
conversations  between  students. 
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Tables 


Student  Responses  to  Question  41: 
What  did  you  like  least  about  this  course? 
(Numbers  are  percentages.) 


Comment 

Live 
(N  =  29) 

VTT 

Local 
(N  =  105) 

VTT 
Remote 
(N  =  118) 

Dry  material 

28 

25 

12 

Audio-visual 

17 

15 

17 

Sound  problems 

0 

4 

19 

Tests/quizzes/homework 

14 

10 

12 

Not  enough  time 

14 

11 

8 

Course  procedures/processes 

0 

7 

10 

Personal 

7 

5 

11 

Course  organization 

7 

10 

3 

Classroom  environment 

14 

9 

3 

Too  much  time 

0 

5 

5 

Table  6 

Student  Responses  to  Question  42: 

Discuss  any  suggestions  you  have  for  improving  how  video  teletraining 

is  used  in  this  course. 

(Number  are  percentages.) 


Comment 

VTT 

Local 
(N  =  49) 

VTT 

Remote 

(N=121) 

Improve  video 

26 

27 

Improve  audio 

14 

28 

Show  otiier  class 

31 

17 

Use  push-to-talk  mikes 

12 

17 

Change  procedures/processes 

16 

13 

26 


Observations 


The  instructors  gained  skill  and  confidence  in  the  VTT  classroom  rapidly.  All  functioned 
effectively  within  a  few  hours  and  became  comfortable  and  skilled  before  the  camera  in  about  2 
weeks. 

The  most  consistent  problem  was  with  audio.  Difficulties  were  encountered  in  setting  mixer 
and  PA  system  levels,  preventing  inadvertent  transmission  of  noise,  and  controlling  feedback.  The 
heavy,  gated  lavalier  microphone  provided  with  the  Shure  mixer  never  operated  satisfactorily  and 
was  eventually  replaced  with  a  lighter,  ungated  lavalier  microphone. 

It  became  obvious  that  there  is  a  trade-off  in  providing  students  with  sound-activated  table 
microphones.  On  the  positive  side,  these  allow  students  to  be  heard  simply  by  speaking;  no  button 
has  to  be  pressed  to  activate  the  microphone.  On  the  negative  side,  they  preclude  the  possibility  of 
private  conversations  between  students  since  everything  the  student  says  will  be  heard,  and  they 
increase  the  likelihood  of  class  disruptions.  Everything  a  student  says  will  be  picked  up  by  his  or 
her  microphone  and  broadcast  over  the  public  address  system  to  the  other  class.  Push-to-talk 
microphones  would  prevent  such  occurrences  and  allow  more  natural  communication  between 
students. 

The  originating  classroom  was  controlled  by  the  instructor  and  the  remote  classroom  by  his  or 
her  proxy,  the  facilitator,  or  in  the  facilitator’s  absence,  the  class  leader.  The  facilitator  handled 
administrative  matters,  maintained  class  order,  operated  equipment,  administered  and  scored  tests 
and  acted  as  the  instructor’s  eyes  and  ears  in  the  remote  class.  Tlie  importance  of  the  facilitator 
became  obvious  during  periods  of  his  absence  when  there  would  often  be  a  lack  of  coordination 
between  classes,  technical  problems  might  go  unresolved,  and  the  remote  class  would  not  run  as 
smoothly.  Moreover,  the  facilitator  became  inert  tsingly  important  as  video  (and  later  audio) 
became  degraded  and  the  instructor  had  less  direct  information  on  what  was  happening  in  the 
remote  class. 

Analysis  of  classroom  interaction  analysis  forms  revealed  that  all  classes  spent  over  95  percent 
of  lecture  time  “on  task’’  (i.e.,  performing  class-related  activities  (Table  D-14)),  Live  classes 
tended  to  discuss,  talk,  or  read  during  lecture  slightly  more  tlian  VTT  local  or  remote  classes.  This 
appears  to  be  a  result  of  VTT  instructors  making  a  sharper  distinction  between  lecture  and 
laboratory/discussion  parts  of  classes  (i.e.,  carefully  separating  different  class  processes  such  as 
lectures,  discussion,  and  laboratories).  During  laboratories  (Table  D-15),  VTT  remote  students 
spent  more  time  talking  with  one  another  and  less  time  discussing  classroom  activities  with  the 
instructor,  but  in  other  respects  classroom  interactions  were  very  similar  in  live  and  VTT  classes. 

In  debrief  interviews,  instructors  expressed  their  preference  for  teaching  in  VTT  classrooms 
that  permitted  2-way  video  rather  than  1-way  video.  It  appeared  to  take  instructors  slightly  longer 
to  adapt  to  1-way  video  than  2- way  video.  Instructors  who  had  been  teaching  in  2-way  video 
classrooms  appeared  to  find  the  first  week  of  teaching  in  1-way  video  classrooms  somewhat 
stressful.  They  may  have  felt  that  their  control  over  the  remote  classroom  was  diminished  because 
they  could  not  see  the  remote  students.  However,  by  the  end  of  the  second  week  of  teaching  in  1- 
way  video  classrooms,  the  instructors  had  developed  new  ways  of  coping  with  the  unseen  class  and 
had  adjusted. 
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Comparisons  Among  Conditions  1, 5, 6,  and  7 

Copclitions  are  compared  below  based  on  (1)  day  3  quiz,  (2)  course  questionnaire,  and  (3) 
obse:vations. 

Day  3  Quiz 

Table  D-16  shows  the  raw  day  3  quiz  scores  for  all  subjects  in  conditions  1, 5, 6,  and  7  in  local 
and  remote  classrooms.  The  data  are  summarized  graphically  in  Figure  13. 
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Figure  13.  Bar  graph  of  raw  day  3  quiz  scores 
for  conditions  1, 5, 6, 7. 
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Student  senioritv  varied  with  condition  (see  Table  D-2)  and  might  influence  perfomiance  on 
the  day  3  quiz  and  mask  the  effects  of  the  independent  variable.  The  data  were  analyzed  using  an 
hierarchical  analysis  of  covariance  (Cohen  &  Cohen,  1975;  Overall  &  Spiegel,  1969).  Seniority, 
the  covariate,  was  found  to  have  a  significant  effect  on  final  examination  performance,  F( 1,358)  = 
4.72,  p  <  .05.  Variance  due  to  the  covariaie  was  removed  in  subsequent  comparisons,  which  were 
made  in  the  following  order 

1.  Live  (condition  1)  versus  All  VTT  (conditions  5, 6, 7). 

2.  Audiographics  (condition  7)  versus  1 V/IA  and  1V/1A+  (conditimis  5, 6). 

3.  IV/IA  (condition  5)  versus  IV/lA-t-  (condition  6). 

4.  4.  Local  classroom  (across  conditions  5, 6, 7)  versus  remote  classroom  (across  conditions 
5,6,7). 
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Table  7  summarizes  the  results  of  this  analysis.  The  amount  of  variance  accounted  for  by 
seniority  was  statistically  significant  but  smaller  than  the  comparable  effect  in  Table  3.  The 
difference  is  probably  the  result  of  the  reduced  reliability  of  the  15-item  quiz  underlying  Table  7 
as  compared  to  the  50-item  final  examination  underlying  Table  3. 

Table? 

Hierarchical  ANCOVA  Results  for  Conditions  1, 5, 6»  7 


Comparison 

Variable  1 

Mean  1 

Variable  2 

Mean  2 

df 

F 

P 

Live  vs.  all  VTT 
(5.6.7) 

Live 

87.24 

All  VTT  (5. 6, 7) 

8235 

1358 

4.81 

<05 

Audiographics  vs. 
1V/1A& 

1V/1A+ 

Audiographics 

82.89 

IV/IA  &  1V/1A+ 

82.04 

1358 

<1 

NS 

IV/IA  vs.  1V/1A+ 

IV/IA 

79.78 

1V/1A+ 

83.10 

1358 

1.89 

NS 

Local  vs.  Remote 

Local 

81.89 

Remote 

82.81 

1358 

<1 

NS 

The  difference  between  Live  and  All  VTT  was  statistically  significant.  Students  in  live  classes 
scored  about  5  points  higher  than  VTT  students,  a  substantial  difference.  None  of  the  remaining 
comparisons  produced  statistically  significant  results.  These  findings  suggest  that  audiographics 
and  1-way  video  are  not  adequate  substitutes  for  live  instruction.  However,  more  refined 
audiographics  and  1-way  video  simulations  might  produce  better  results. 

Course  Questionnaire 

Students  in  conditions  5,  6,  and  7  completed  the  same  questionnaire  as  students  in  conditions 
1 , 2, 3,  and  4.  However,  conditions  5, 6,  and  7  occurred  only  during  day  2,  so  comments  reflect  the 
surrounding  treatment  (equivalent  to  condition  4),  which  occurred  on  days  1,  3,  and  4  as  well  as 
day  2's  experimental  treatment.  Nonetheless,  many  student  comments  made  clear  and 
unambiguous  references  to  day  2’s  experimental  treatment  in  responding  to  question  41  (student 
dislikes),  which  are  revealing.  Four  students  in  condition  5  and  five  in  condition  6  stated  that  what 
they  disliked  was  not  being  able  to  speak  to  the  insuucior.  Three  students  in  condition  7  disliked 
not  being  able  to  sec  the  instructor. 

Observations 

The  instructors  became  more  proficient  with  practice,  but  one  of  the  instructors  never  appeared 
comfortabL  while  teaching  in  research  conditions  5,  6,  and  7  and  expressed  dissatisfaction  with 
the  VTT  method  of  instructional  delivery. 

The  effects  of  audio  problems  w'crc  compounded  when  students  in  the  receiving  classroom 
were  unable  to  communicate  with  the  instructor.  There  were  a  few  occasions  when  audio  problems 
made  it  difficult  to  understand  the  instnictor  but,  since  there  w^as  no  way  to  communicate  with  him 
or  her,  no  corrective  action  could  be  taken. 
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Students  expressed  Ihistration  during  conditions  S  and  6  when  they  were  unable  to  speak  to  the 
instructor.  We  observed  an  increase  in  the  noise  level  in  the  classroom  as  students  began  addressing 
questions  to  one  another  and  carrying  on  discussions  in  class  during  the  lecture.  In  addition, 
students  were  observed  to  take  greater  liberties  in  terms  of  moving  in  and  out  of  the  classroom, 
taking  breaks,  performing  written  exercises  and,  in  general,  disconnecting  somewhat  from  the 
originating  classroom. 

Analysis  of  classroom  interaction  analysis  forms  revealed  that  all  classes  spent  over  90  percent 
of  lecture  time  “on  task”  (i.e.,  performing  class-related  activities  (Table  D-17)).  Live  classes 
tended  to  discuss  or  read  during  lecture  slightly  more  than  VTT  local  or  remote  classes.  During 
laboratories  (Table  D-18),  VTT  remote  students  spent  more  time  “off  task”  than  students  in  live 
classes.  Students  in  conditions  5, 6,  and  7  also  spent  more  time  off  task  than  students  in  conditions 
2,  3,  and  4  (see  Table  D-15).  The  percentage  of  students  off-task  in  conditions  5,  6,  and  7  was 
highest  for  remote  classes  but  local  classes  were  also  affected.  Conditions  5, 6,  and  7  limited  the 
amount  and  quality  of  communication  between  students  and  instructor,  and  this  evidently  made 
group  activities  more  difficult  to  manage,  resulting  in  inefficient  use  of  class  time.  Informal  breaks 
were  permitted  during  the  laboratories,  and  when  2- way  communication  was  limited,  tlie  instructor 
appeared  to  have  difficulty  controlling  student  presence  in  the  classroono;  students  were  absent 
from  the  room  a  relatively  high  percentage  of  time  in  VTT  conditions. 

DISCUSSION 

The  results  are  based  on  6  months  of  data  collection  with  a  large  number  of  subjects,  but  with 
a  single  lecture-based  Navy  training  course.  The  results  should  generalize  to  lecture-based  courses 
and  to  the  lecture  and  discussion  portions  of  other  types  of  Navy  courses.  The  results  indicate  that 
VTT  in  several  different  fomis  was  effective  both  in  terms  of  student  performance  and  student  and 
instructor  acceptance.  Type  of  VTT  technology  did  influence  student  performance  and  attitudes, 
but  had  a  far  smaller  effect  Uian  seniority.  Instructors  had  definite  preferences  for  certain  VTT 
technologies,  but  these  did  not  appear  to  impact  their  classroom  perfomiance. 

The  most  successful  VTT  technologies  were  fully-intcraciive;  2V/2A,  1V/2A. 

Student  performance  was  comparable  with  live  and  VTT  instruction  using  2V/2A+,  2V/2A,  and 
1 V/2A.  Using  2-way  video  docs  not  appearto  improve  student  performance  as  compared  to  1-way 
video,  but  instructors  prefer  2- way  video  and  siudctiis  expressed  the  desire  to  sec  their  cohorts  in 
other  classes,  which  requires  2- way  video. 

The  least  successful  VIT  technologies  were  partially-intcraciivc:  IV/IA,  IV/lA-f, 
audiographics.  Student  test  performance  was  poorer  with  VTl'  systems  that  Festricicd  remote 
students'  ability  to  converse  with  or  sec  the  instructor  and  the  perfonnanee  deae^nrcni  was  evident 
in  both  local  and  remote  c!as.srooms.  Evidence  suggests  that  student  acceptance  of  partially- 
intcraciivc  VTT  was  lower  than  with  fully-inicractivc  VTT.  Similar  results  would  be  expected  with 
videotaped  instruction  because  of  its  similarity  to  1 V/I  A.  Students  adapted  to  compensate  fctf  the 
shortcomings  of  VTT  technologies  they  were  trained  wnih.  For  example,  when  students  in  remote 
classrooms  wci '  unable  to  speak  to  the  instructor,  they  increased  their  level  of  interaction  with  one 
another.  Observing  their  compensatory  behavior  reminded  us  that  much  of  Navy  training  occurs 
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under  difficult  conditions  and  that  Navy  personnel  are  generally  tolerant  and  Gy  to  make  the  best 
of  the  situation. 

Partially-interactive  technologies  did  not  work  as  well  as  fully-interactive  technologies,  but 
they  did  work;  student  test  performance  decrements  resulting  from  the  use  of  partially-interactive 
technologies  were  small.  Because  these  technologies  cost  so  much  less  than  fully-interactive 
technologies,  they  should  be  considered  for  use  where  cost  is  a  major  consideration. 

Our  observations  suggest  that  good-quality  audio  is  more  critical  to  training  success  than  good 
quality  video;  the  student  can  learn  from  a  lecture  without  observing  the  instmctor,  br  t  can  seldom 
learn  from  observing  without  hearing  the  lecture.  The  most  serious  shortcoming  of  the  VTT 
systems  we  simulated  was  audio.  The  audio  problem  is  multi-dimensional  and  has  classroom 
design,  hardware  selection,  and  procedural  elements.  Additional  work  needs  to  be  done  to  refine 
the  audio  systems  and  procedures  used  in  VTT. 

RECOMMENDATIONS 

1 .  The  Chief  of  Naval  Education  and  Training  and  the  Naval  Education  and  Training  Program 
Management  Support  Activity  should  continue  efforts  to  refine  the  CNET  VTT  network. 

2.  The  Chief  of  Naval  Education  and  Training  and  the  Naval  Education  and  Training  Program 
Management  Support  Activity  should  analyze  the  feasibility  and  cost-effectiveness  of  extending 
the  architecture  of  the  CNET  VTT  network  using  VTT  technologies  such  as  1-way  video  with  2- 
way  audio  and  1-way  video  with  ,-way  audio. 

3.  The  Chief  of  Naval  Education  and  Training  and  the  Naval  Education  and  Training  Program 
Management  and  Support  Activity  should  originate  Problem  Description  and  Need  Justifications 
to  corxtinue  the  investigation  of  the  applicability  of  VTT  beyond  lecture-based  courses  (e.g.,  in 
courses  using  “hands-on”  laboratories,  small-group,  and  other  training  processes). 
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APPENDIX  A 
STUDENT  SURVEY 


A-0 


STUDENT  SURVEY 

ADMINISTRATION  AND  OPERATION  OF  SHIPBOARD  3M  SYSTEMS 

1-500-0025 


Date: _  Rate _  Room  No. _ 

Name _ SSN(opt) _ 

PLEASE  CIRCLE  THE  LETTER  FOR  THE  MOST  APPROPRIATE  ANSWER 


1.  Did  you  truest  this  course  of  instruction? 

A.  Yes  B.  No 

2.  Have  you  attended  this  course  within  the  past  2  years? 

A.  Yes  B.  No  If  yes,  where? _ 

3.  Have  you  had  any  previous  SNAPU  experience? 

A.  Yes  B.  No 

4.  What  is  your  current  3M  PQS  qualification  level? 

a.  Maintenance  Person  (3M-301) 

b.  Work  Center  Supervisor  (3M-302) 

c.  Division  Officer  (3M-303) 

d.  Departmental  Assistant  (3M-304) 

e.  Department  Head  (3M-305) 

f.  3M  Coordinator  {3M-306) 

g.  None 

5.  What  is  your  current  assignment? 

a.  Maintenance  Person 

b.  Work  Center  Supervisor 

c.  Group  Supervisor 

d.  Di  vision  Officer 

e.  Departmental  3M  Assistant 


e.  Department  Head 

f.  3M  coordinator 

g.  3M  inspector 

h.  None 


6.  What  position  will  you  be  going  to  upon  completion  of  this  course? 


a.  Maintenance  Person 

b.  Work  Center  Supervisor 
e.  Group  Supervisor 

d.  Division  Officer 

e.  Departmental  3M  Assistant 


e.  Department  Head 

f.  3M  coordinator 

g.  3M  inspector 

h.  Unknown 


FOR  OFFICE  USE  ONLY 

Quiz  One _ 

Quiz  Two _ 

Quiz  Three _ 

FINAL  GRADE 


- 1  - 
A-1 


APPENDIX  B 


STUDENT  COURSE  EVALUATION  QUESTIONNAIRE 


}.  Nams 


2.  Today’s  date 


Fitsf 

/ - /- 


Ml 


Last 


Rate 


3.  Course  title  Q  a.  3M  AdmIn/Ops 
I  I  b.  Safety  Offleer 
EH  c.  Safety  Petty  Officer 
I  I  d.  Other  (specify) _ 

Section  1:  Course  Evaluation 


On  the  last  page,  there  Is  room  to  comment 
on  specific  areas  of  concern.  Please  use  this 
space  to  clarify  how  Improvement  can  be 
made. 


For  each  of  the  following  statements  ( 1  through  2 1).  check  the  appropriate  bo:- 

corresponding  to  a  scale  of 

1  (unsatisfactory)  through  5  (outstanding),  with  3  being  average.  Leave  any  statements  that  do  not  apply  to 
this  course  blank. 

Instructor 

Unsatisfactory - ► 

(Average  =  3) 

■  Outstanding 

1 .  Instructor  prepared  for  class . 

..  iG 

40 

a 

2.  Instructor  presented  lessons  clearly . 

..  ,□ 

2G 

c 

40 

a 

3.  Instructor  answered  student  questions . 

„  ,□ 

20 

n 

40 

30 

4.  Instructor  encouraged  class  participation . 

.  iG 

20 

30 

40 

30 

5.  Instructor  was  available  for  individual  assistance  outside  of  class... 

20 

a 

40 

30 

6.  Instructor  treated  students  fairly . 

20 

30 

40 

30 

Audio-Visual  Aids 

7.  Video  screen  was  large  enough  to  be  seen . 

..  n 

20 

30 

40 

30 

8.  Video  screen  was  close  enough  to  be  seen . 

..  ,□ 

20 

30 

40 

30 

9.  Image  on  video  screen  was  cleor . 

..  ,□ 

20 

30 

40 

30 

10.  Audio  transmission  was  loud  enough  to  hear  Instructor's  voice . 

.  .□ 

20 

30 

40 

30 

1 1 .  Audio  transmission  was  clear  enough  to  hear  whoi  the  instructor  said.,  i  EH 

20 

30 

40 

30 

12.  Graphics/Slides/Transparencles  on  TV  were  readable . 

.  iG 

20 

30 

40 

30 

13.  Television  was  in  working  order . 

......  G 

20 

30 

40 

30 

14.  Your  microphone  was  in  workirrg  order . 

..  .G 

2O 

30 

40 

30 

Tests/Homewofk 

15.  Test  questions  were  clearly  written . 

,.  .G 

2O 

30 

40 

30 

16.  Test  questions  were  directly  uated  to  course . 

.G 

20 

30 

40 

30 

1 7.  Test  onswers  were  graded  fairly . 

.  .G 

2D 

30 

4EH 

30 

18.  Homework  assignments  were  understandable . 

.G 

20 

30 

40 

30 

19.  Homework  assignments  were  directly  related  to  course . 

.  .G 

20 

30 

40 

30 

Overall 

20.  Comparison  of  this  Instructor  to  other  Navy  instructors  that  hove 
taught  you  in  the  past . 

..  '□ 

20 

30 

40 

30 

2 1  Comparison  of  this  course  to  other  Na\/y  courses  that  you  have 
token  in  the  post . 


,□  20  30  o  sD 
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Section  2:  instructor-student  interaction 


22.  Did  you  talk  to  the  Instructor  or  ask  any  questions  during  the  regular  hours  of  this  course? 

□  a.  Yes 

□  b.No 

23.  How  did  the  video  tele-tralning  method  of  Instruction  affect  your  opportunities  to  talk  to  the  Instructor  or 
ask  questions,  as  compared  to  traditional  methods  of  Instrucfion? 

[~|  a.  More  opportunities 

n  b.  No  effect  on  opportunities 

n  c.  Fewer  opportunities 

24.  Were  there  adequate  opportunities  for  remedial  irtstructlon  outside  of  the  regular  hours  of  this  course? 

O  a.Yes 

□  b.No 

r~]  c.  Remedial  instruction  was  not  necessary  for  this  course. 

25.  Did  you  attend  any  remedial  instruction  periods? 

Q  a.Yes 
n  b.No 

Answer  questions  26  and  27  only  U  you  answered  vesio  question  25, 

26.  From  whom  did  you  obtain  the  remedial  Instruction? 

n  a.  Instructor  via  video  tele-training 
n  b.  Instructor  who  was  physically  present  In  the  same  room 

□  c.  Other  (specify) _ 

27.  How  may  hours  of  remedial  iristructlon  did  you  receive? _ 

Section  3:  Student  Comments 

28.  Which  method  of  instruction  would  you  have  preferred  for  this  course? 

r~l  a.  Video  tele-tralning  where  instajctor  Is  on  TV 

r~l  b.  Traditional  methods  of  instruction  where  instructor  is  physically  present  in  the  classroom 
n  c.  Indifferent  between  video  tele-training  and  traditional  methods  of  Instruction 

29.  How  did  the  participation  of  students  at  other  sIteCs)  affect  you  learning  during  this  course? 

n  a.  Improved  learning 
r~l  b.  No  effect  on  learning 
(~1  c.  Reduced  learning 
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Section  3:  Overall  Impressions 


Content  of  Course: 

DIRECTIONS:  Thinking  only  of  the  content  of  the  course,  place  an  X  In  the  one  spoce  of  the  seven  between 
each  adjective  pair  that  best  describes  your  op/n/on  of  the  content  of  this  course.  The  closer  you  place 
your  X  toward  one  adjective  or  the  other,  the  more  you  tNrdc  the  adjective  better  describes  the  course 
content  than  the  other. 


EXAMPLE:  If  you  think  the  content  of  this  course  was  very  meaningful,  you  would  f^ce  an  X  In  the 
space  closest  to  that  adjective. 

meaningful  I  I  I  I  I  I  i  I  meaningless 


CONTENT  OF  THIS  COURSE 

30.  interesting  j _ j _ j _ j _ j _ j _ j _ j  boring 

31.  important  I  I  I  I  I  I  I  I  unimportant 

32.  powerful  ^ _ | _ j _ j _ I  I _ j _ I  vtreak 

33.  worthless  I  I  I  I  I  I  I  I  valuable 

34.  good  I  I  I  I  I  I  II  bad 


Form  of  Instruction: 

DIRECTIONS:  Now  thinking  only  of  the  form  of  Instruction,  place  an  X  In  the  one  space  of  the  seven  between 
each  adjective  pair  that  best  describes  what  you  think  the  form  of  Instructigr)  was  like  In  thb  course.  The 
closer  you  place  the  X  toward  one  adjective  or  the  other,  the  more  you  think  the  adjective  belter  describes 
the  form  of  Instruction  than  the  other. 


FORM  OF  INSTRUCVON 


35. 

aood  1  1 

1  1 

1 

1  1  bod 

36. 

weak  1  1 

1  1 

1 

1  1  powerful 

37. 

annovina  1  1 

1  1 

1 

1  1  pleasing 

38. 

successful  1  1 

1 

1  1  unsuccessful 

39. 

• 

neaative  1  1 

1  1 

1 

1  1  positive 

VIT  Form  1 
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Section  3:  Overall  Impressions  (continued) 


40.  What  did  you  like  most  about  this  course? 


41 .  What  did  you  iike  least  about  this  course? 


42.  Discuss  any  suggestions  thot  you  have  for  improving  how  video  tele-training  Is  used  In  this  course. 
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Section  4:  Safety 

43.  Did  lessons  Include  safety  where  applicable  (Safety  as  it  applies  to  your  job)?. 

□  Yes 

*  □  No 

n  See  Remarks 

44.  Did  instructor(s)  adequately  cover  safety  Items  prior  to  conducting  performance  labs? 

□  Yes 
«  □  No 

□  See  Remarks 

45.  Was  safety  a  primary  consideration  of  the  instructor(s)? 

□  Yes 

*  □  No 

□  See  Remarks 

46.  Was  the  classroom/laboratory  equipment  always  safe  for  use? 

□  Yes 

*  □  No 

I  I  See  Remarks 


*  Please  comment  in  space  provided  below.  If  more  space  Is  needed,  please  use  additional  sheets. 
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Section  3:  Overall  impressions  (continued) 

43.Comment  on  usefulness  and  adequacy  of  the  course  content:. 


Section  4:  Safety 

47.  Did  lessons  include  safety  where  applicable  (Safety  as  applies  to  your  Job)?. 


n  Yes  n  No  Q  See  Remarks 

48.  Did  instructor(s)  adequately  cover  safety  items  prior  to  conducting  performance  lobs? 

r~|  Yes  n  D  See  Remarks 

49.  Was  safety  a  primary  consideration  of  the  IrctructorCsi)? 

(71  Yes  Q  No  (71  SoeRemcuks 

50.  Wee  the  ciossroom/laboratory  equipment  oJways  safe  for  use? 

D  [D  Q  See  Remarks 

Comment  on  safety. _ 
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CLASSROOM  INTERACTION  ANALYSIS  FORM 


C-0 


Classroom  Interaction  Analysis  Form 

R&cofd  5  Studonis  at  2  mlnuto  Inton/ab 


ON  TASK 

OFF  TASK 
« 

Discussion 

Time  l->Ss 

1*0 

Talking  Reading 

SmS  Lesson 

Questions 

Other 

(D&scO 

Talk 

_ 

’Day 

Dream’ 

Other 

(OescJ 

Date  _ 

Observer 


Oass 


f  Rrjt  Deck 


□  se. 


orvd  Deck 


APPENDIX  D 
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ADDITIONAL  TABLES 


Table  D-l 

Class  Size  as  a  Function  of  Research  Condition  and  Classroom 


Research 

Condition 

Local 

Mean 

Local 

Total  N 

Remote 

Mean 

Remote 
Total  iV 

1 

19.7 

59 

N/A 

N/A 

2 

20.0 

60 

19.0 

57 

3 

20.3 

61 

20.3 

61 

4 

22.0 

66 

21.7 

65 

5 

16-0 

32 

15.5 

31 

6 

22.3 

67 

22.7 

68 

1 

18.7 

56 

20.0 

60 

Table  D-2 


Class  Seniority  as  a  Function  of  Research  Condition  and  Classroom 


Research 

Condition 

Local 

Mean 

Local 

SD 

Remote 

Mean 

Remote 

SD 

1 

2.42 

1.12 

N/A 

N/A 

2 

2.53 

1.33 

2.53 

1.15 

3 

2.28 

1.13 

2.67 

1.15 

4 

2.5) 

i.Oi 

2.40 

5 

2.28 

0.89 

2.10 

0.94 

6 

2.24 

1.18 

2.40 

1.05 

7 

2.43 

1.13 

2.42 

1.08 

D-l 


Table  D-3 


Raw  Final  Examination  Scores  for  Conditions  1, 2, 3, 4 
Class  Seniority  as  a  Function  of  Research  Condition  and  Classroom 


Research 

Condition 

Local 

Mean 

Local 

SD 

Remote 

Mean 

Remote 

SD 

Live 

90.03 

7.53 

N/A 

N/A 

2V/2A+ 

87.56 

7.89 

87.23 

10.35 

2V/2A 

88.10 

9.17 

86.89 

7.65 

1V/2A 

89.91 

5.66 

88.25 

7.51 

Table  D>4 

Student  Attitude  Measures  on  Statements  Relating  to  Instructor  (Scale  1-5) 

Question  Live 

2V/2A+ 

Lflcal 

2V/2A+ 

Remote 

2V/2A 

Local 

2V/2A 

Remote 

1V/2A 

Local 

1V/2A 

Remote- 

1.  Instructor  prepared 

for  class,  4.7 

2.  Instructor  presented 

4.5 

4.5 

4.6 

4.5 

4.7 

4.7 

lessons  clettrly.  4.5 

3.  Instructor  answered 

4.4 

4.3 

4.5 

4.3 

4.6 

4.5 

student  questions,  4.7 

4.  Instructor  encouraged 

4.6 

4.4 

4.6 

4.5 

4.6 

4.7 

student  participation.  4.7 

5.  lastructcr  was  avail¬ 
able  for  individual 
a,ssistantx  outside 

4.5 

4.3 

4.6 

4.5 

4.5 

4.6 

of  class.  4.3 

6.  Instructor  trcfiled 

4,4 

4.1 

4.3 

4.0 

4.6 

4.5 

.studeni.s  fairly.  4." 

4.7 

4.5 

4.6 

4.5 

4.7 

4.7 

Table  D-5 

Student  Attitude  Measures  on  Statements  Relating  to  Audiovisual  Aids  (Scale  1-5) 


Question 

Live 

2v'/2A+ 

Local 

2V/2A+ 

Remote 

2V/2A 

Local 

2V/2A 

Remote 

1V/2A 

Local 

1V/2A 

Remote 

• 

7.  Video  screen  was  large 
enough  to  be  seen. 

4.4 

4.5 

4.4 

4.4 

4.4 

4.6 

4.7 

8.  Video  screen  was  close 
enough  to  be  seen. 

4.3 

4.5 

4.4 

4.4 

4.4 

4.6 

4.6 

9.  Image  on  video  screen 
was  clear. 

4.0 

4.3 

4.4 

4.0 

4.2 

4.4 

4.7 

lO.Audio  transmission 
was  loud  enough  to 
hear  instructor’s 
voice. 

N/A 

4.7 

4.4 

4.3 

3.6 

4.6 

4.4 

11. Audio  transmission 
was  clear  enough  to 
hear  what  the  in¬ 
structor  said. 

N/A 

4.7 

4.3 

4.4 

3.7 

4.6 

4.5 

12.Graphics/Slides/Trans- 
parencies  on  TV  were 
readable. 

4.1 

4.3 

4.4 

4.4 

4.3 

4.4 

4.6 

13. Television  was  in 
working  order. 

4.7 

4.7 

4.6 

4.5 

4.5 

4.6 

4.7 

14. Your  microphone 
was  in  working  order. 

N7A 

4.7 

4.5 

4.4 

4.2 

4.7 

4.7 

Table  D-6 


Student  Attitude  Measures  on  Statements  Relating  to  Tests  and  Homework  (Scale  1>5) 


Question 

Live 

2V/2A-(- 

Local 

2V/2A-H 

Remote 

2V/2A 

Local 

2V/2A 

Remote 

1V/2A 

Local 

ivy2A 

Remote 

IS.Test  questions  were 
clearly  written. 

3.9 

4.0 

3.9 

4.0 

3.7 

4.1 

4.4 

16.Tesi  questions  were 
directly  related  to 
course. 

4.4 

4.5 

4.4 

4.3 

4.3 

4.5 

4.6 

n.Test  answers  were 
graded  fairly. 

4.5 

4.5 

4.2 

4.5 

4.3 

4.5 

4.7 

IS.Homework  assignments 
were  understandable. 

4.2 

4.2 

4.0 

4.3 

4.1 

4.3 

4.3 

19.Homework  assignments 
were  directly  related 
to  course. 

4.3 

4.4 

4.3 

4.3 

4.2 

4.4 

4.5 

Table  D-7 

Student  Attitude  Measures  on  Statements  Relating  to  Overall  Assessment 
of  Instructor  and  Course  (Scale  1*5) 

Question 

Live 

2V/2A+ 

Local 

2V/2A+ 

Remote 

2V/2A 

Local 

2V/2A 

Remote 

1V/2A 

Local 

1V/2A 

Remote 

20.Comparison  of  in¬ 
structor  to  others 
that  have  taught  you 

111  the  past. 

4.2 

4.3 

4.0 

4.3 

4.1 

4.3 

4.3 

21.CQmpari.son  of 
this  course  to  other 

Navy  courses  that 
you  have  taken  in 
the  past. 

3.9 

4.1 

3.9 

4.0 

4.0 

4.1 

4.0 

D4 


Table  D-8 


Semantic  Differential  Responses  Relating  to  Course  Content  (Scale  1-7) 


Question 

Live 

2V/2A+ 

Local 

2V/2A+ 

Remote 

2V/2A 

Local 

2V/2A 

Remote 

1V/2A 

Local 

1V/2A 

Remote 

30.(interest) 

5.1 

4.8 

4.6 

5.3 

5.3 

5.6 

4.7 

31. (importance) 

6.6 

6.5 

6.1 

6.5 

6.5 

6.6 

6.3 

* 

32.  (power) 

5.4 

5.3 

4.9 

5.5 

5.4 

5.5 

5.1 

33.(value) 

6.1 

6.1 

5.8 

6.4 

6.3 

6.4 

6.1 

V 

34.(goodness) 

6.3 

6.2 

5.9 

6.3 

6.3 

6.4 

6.1 

Table  D-9 

Semantic  Differential  Responses  Relating  to  Form  of  Instruction  (Scale  1-7) 


Question 

Live 

2V/2A+ 

Local 

2V/2A+ 

Remote 

2V/2A 

Local 

2V/2A 

Remote 

1V/2A 

Local 

ivy2A 

Remote 

35.(gooclness) 

6.3 

5.9 

5.8 

6.3 

6.0 

6.4 

6.1 

36.  (strength) 

5.6 

5.4 

5.0 

5.9 

5.5 

5.7 

5.2 

37.(plcasantncss) 

5.6 

5.2 

4.9 

5.6 

5.4 

5.7 

5.2 

38.(succcss) 

6.1 

5.8 

5.7 

6.3 

6.0 

6.2 

5.8 

39.(positivencss) 

6.1 

5.9 

5.6 

6.2 

6.0 

6.2 

5.8 

Table  D-10 


Student  Responses  to  Question  22: 

Did  you  talk  to  the  instructor  or  ask  any  questions  during  the  regular  hours  of  this  course? 

(Numbers  are  percentages.) 


Response 

Live 

2V/2A+ 

Local 

2V/2A+ 

Rcjnote 

2V/2A 

Local 

2V/2A 

Remote 

IVY2A 

Ivocal 

1V/2A 

Remote 

Yes 

91.2 

90.6 

75.8 

88.3 

83.6 

83.1 

70.2 

No 

8.8 

9.4 

24.2 

11.7 

16.4 

16.9 

29.8 
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Table  D-ll 


Student  Responses  to  Question  23: 

How  did  the  VTT  method  of  instruction  affect  your  opportunities  to  talk  to  the  instructor 
or  ask  questions,  as  compared  to  traditional  methods  of  instruction? 

(Numbers  are  percentages.) 


Response 

2V/2A+ 

Local 

2Vy2A+ 

Remote 

2V/2A 

Local 

2V/2A 

Remote 

1V/2A 

Local 

1V/2A 

Remote 

More  opportunities 

24.2 

25.4 

20.3 

11.5 

30.9 

10.5 

No  effect  on  opportunities 

72.6 

68.3 

76.3 

65.6 

69.1 

64.9 

Fewer  opportunities 

3.2 

6.3 

3.4 

23.0 

0.0 

24.6 

Table  D-12 

Student  Responses  to  Question  28: 

Which  method  of  instruction  would  you  have  preferred  for  this  course? 

(Numbers  are  percentages.) 


Response 

2V/2A+ 

Local 

2V/2A+ 

Remote 

2V/2A 

Local 

2V/2A 

Remote 

1V/2A 

Local 

1V/2A 

Remote 

VTT 

17.7 

41.3 

20.7 

18.3 

12.5 

27.3 

Traditional 

33.9 

20.6 

25.6 

30.0 

33.9 

32.7 

No  preference 

48.4 

38.1 

53.4 

51.7 

53.6 

40.0 

Table  D-13 


Student  Responses  to  Question  29: 

How  did  the  participation  of  student  at  other  sitc(s)  affect  your 
learning  during  this  course? 

(Numbers  are  percentages) 


Response 

2V/2A+ 

Local 

2V/2A+ 

Remote 

2V/2A 

Local 

2V/2A 

Remote 

1V/2A 

Local 

ivy2A 

Remote 

Improved  learning 

22.2 

22.2 

32.7 

27.9 

39.3 

25.0 

No  effect  on  learning 

74.6 

71.4 

63.9 

58.9 

69.6 

Reduced  learning 

3.2 

6.3 

8.2 

1.8 

5.4 
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Table  D-14 

Interaction  Analysis  Summary  for  Lectures  (Conditions  1, 2, 3, 4) 


AUVTT 

AUVTT 

Events 

Live 

Local 

Remote 

On-task 

Lecture 

91.9 

99.2 

99.4 

Discussion 

3.3 

0.0 

0.1 

Talking 

0.7 

0.0 

0.0 

Reading 

2.2 

0.0 

0.0 

Other 

0.0 

0.0 

0.0 

Subtotals 

98.1 

99.2 

99.5 

Off-task 


Talking 

0.9 

0.4 

0.1 

Daydreaming 

0.3 

Absent  room 

0.1 

0.1 

Subtotals 

2.0 

0.8 

t 
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Table  D-15 


Interaction  Analysis  Summary  for  Labs  (Conditions  1, 2, 3, 4) 


Events 

Live 

AllVTT 

Local 

AllVTT 

Remote 

On-task 

Lecture 

0.2 

0.0 

0.0 

Discussion 

13.0 

3.7 

3.9 

Talking 

33.2 

38.7 

51.8 

Reading 

38.5 

43.6 

28.8 

Other 

1.1 

0.0 

0.0 

Subtotals 

86.0 

86.0 

84.5 

Off-task 

Events 

Talking 

2.9 

2.5 

1.2 

Daydreaming 

8.4 

6.3 

5.3 

Absent  room 

2.9 

5.2 

9.1 

Subtotals 

14.2 

14.0 

15.6 

Table  D-16 

Raw  Day  3  Quiz  Scores  for  Conditions  1, 5, 6, 7 

Research 

Conditions 

Local 

Mean 

Local 

SD 

Remote 

Mean 

Remote 

SD 

Live 

87.24 

12.63 

N/A 

N/A 

iV/lA 

79.35 

16.27 

80.21 

17.17 

1V/1A+ 

81.41 

17.05 

84.82 

14.69 

Audiographics 

83.88 

12.61 

81.95 

16.14 
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Table  D-17 


Interaction  Analysis  Summary  for  Lectures  (Conditions  1, 5, 6, 7) 


Events 

Live 

All  VTT 

Local 

AUVTT 

Local 

On-task 

Lecture 

91.9 

98.5 

95.3 

Discussion 

3.3 

0.0 

0.0 

Talking 

0.7 

0.0 

2.9 

Reading 

2.2 

0.0 

0.1 

Other 

Subtotals 

98.1 

98.5 

98.3 

Off-task 


Talking 

0.9 

0.1 

0.3 

Daydreaming 

1.0 

0.7 

0.1 

Absent  room 

0.1 

0.7 

1.3 

Subtotals 

2.0 

1.5 

1.7 
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Table  D-18 


Interaction  Analysis  Summary  for  Labs  (Conditions  1, 5, 6, 7) 


AllVTT 

AUVTT 

Events 

Live 

Local 

Local 

On-task 

Lecture 

0.2 

0.0 

0.0 

Discussion 

13.0 

7.4 

3.2 

Talking 

33.2 

31.0 

33.7 

Reading 

38.5 

41.2 

36.5 

Other 

1.1 

0.0 
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Off-task 

Talking 
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3.2 

6.5 

Daydreaming 

8.4 

4.1 

7.6 

Absent  room 

2.9 
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12.5 

Subtotals 

14.2 

20.3 

26.6 
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